[GV-115] Enable multiple engines in CI with Example Project #76
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| # Download the unreal-engine.version file from the GDK repo so we can run the example project builds on the same versions the GDK was run against | ||
| # This is not the pinnacle of engineering, as we rely on GitHub's web interface to download the file, but it seems like GitHub disallows git archive | ||
| # which would be our other option for downloading a single file | ||
| GDK_BRANCH_LOCAL="${GDK_BRANCH:-master}" | ||
| NUMBER_OF_TRIES=0 | ||
| while [ $NUMBER_OF_TRIES -lt 5 ]; do | ||
| CURL_TIMEOUT=$((10<<NUMBER_OF_TRIES)) | ||
| NUMBER_OF_TRIES=$((NUMBER_OF_TRIES+1)) | ||
| echo "Trying to download unreal-engine.version from GitHub's UnrealGDK repository, try: $NUMBER_OF_TRIES, timeout: $CURL_TIMEOUT seconds ..." | ||
| curl -L -m $CURL_TIMEOUT https://raw.githubusercontent.com/spatialos/UnrealGDK/$GDK_BRANCH_LOCAL/ci/unreal-engine.version -o ci/unreal-engine.version | ||
| if [ $? -eq 0 ]; then | ||
| break | ||
| fi | ||
| done | ||
|
|
||
| # This script generates BuildKite steps for each engine version we want to test against. | ||
| # We retrieve these engine versions from the unreal-engine.version file in the UnrealGDK repository. | ||
| # The steps are based on the template in nightly.template.steps.yaml. | ||
|
|
||
| if [ -z "${ENGINE_VERSION}" ]; then | ||
| echo "Generating build steps for each engine version listed in unreal-engine.version" | ||
| STEP_NUMBER=1 | ||
| IFS=$'\n' | ||
| for commit_hash in $(cat < ci/unreal-engine.version); do | ||
| REPLACE_STRING="s|ENGINE_COMMIT_HASH_PLACEHOLDER|$commit_hash|g; s|STEP_NUMBER_PLACEHOLDER|$STEP_NUMBER|g" | ||
| sed $REPLACE_STRING ci/nightly.template.steps.yaml | buildkite-agent pipeline upload | ||
| STEP_NUMBER=$((STEP_NUMBER+1)) | ||
| done | ||
| # We generate one build step for each engine version, which is one line in the unreal-engine.version file. | ||
| # The number of engine versions we are dealing with is therefore the counting variable from the above loop minus one. | ||
| STEP_NUMBER=$((STEP_NUMBER-1)) | ||
| buildkite-agent meta-data set "engine-version-count" "$STEP_NUMBER" | ||
joshuahuburn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| else | ||
| echo "Generating steps for the specified engine version: $ENGINE_VERSION" | ||
| sed "s|ENGINE_COMMIT_HASH_PLACEHOLDER|$ENGINE_VERSION|g" ci/nightly.template.steps.yaml | buildkite-agent pipeline upload | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| common: &common | ||
| agents: | ||
| - "agent_count=1" | ||
| - "capable_of_building=gdk-for-unreal" | ||
| - "environment=production" | ||
| - "machine_type=quad" | ||
| - "permission_set=builder" | ||
| - "platform=windows" | ||
| - "scaler_version=2" | ||
| - "queue=${CI_WINDOWS_BUILDER_QUEUE:-v4-2019-11-07-bk3700-fbffad576b9676d7}" # Has FASTbuild disabled | ||
| timeout_in_minutes: 60 # TODO(ENG-548): reduce timeout once agent-cold-start is optimised. | ||
| retry: | ||
| automatic: | ||
| # This is designed to trap and retry failures because agent lost connection. Agent exits with -1 in this case. | ||
| - exit_status: -1 | ||
| limit: 3 | ||
| plugins: | ||
| - ca-johnson/taskkill#v4.1: ~ | ||
|
|
||
| steps: | ||
| - label: "build-and-deploy-:windows:-ENGINE_COMMIT_HASH_PLACEHOLDER" | ||
| command: "powershell -NoProfile -NonInteractive -InputFormat Text -Command ./ci/setup-and-build.ps1" | ||
| <<: *common # This folds the YAML named anchor into this step. Overrides, if any, should follow, not precede. | ||
| artifact_paths: | ||
| - "UnrealEngine/Engine/Programs/AutomationTool/Saved/Logs/*" | ||
| env: | ||
| ENGINE_COMMIT_HASH: "ENGINE_COMMIT_HASH_PLACEHOLDER" | ||
| STEP_NUMBER: "STEP_NUMBER_PLACEHOLDER" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| . "$PSScriptRoot\common.ps1" | ||
|
|
||
| $gdk_branch_name = Get-Env-Variable-Value-Or-Default -environment_variable_name "GDK_BRANCH" -default_value "master" | ||
| $launch_deployment = Get-Env-Variable-Value-Or-Default -environment_variable_name "START_DEPLOYMENT" -default_value "true" | ||
| $engine_version_count = buildkite-agent meta-data get "engine-version-count" | ||
| $project_name = buildkite-agent meta-data get "project-name" | ||
| $gdk_commit_hash = buildkite-agent meta-data get "gdk_commit_hash" | ||
|
|
||
| # Send a Slack notification with a link to the new deployment and to the build. | ||
| Start-Event "slack-notify" "slack-notify" | ||
| # Build Slack text | ||
| if ($env:BUILDKITE_NIGHTLY_BUILD -eq "true") { | ||
| $slack_text = ":night_with_stars: Nightly build of *Example Project* succeeded." | ||
| } else { | ||
| $slack_text = "*Example Project* build by $env:BUILDKITE_BUILD_CREATOR succeeded." | ||
| } | ||
|
|
||
| # Read Slack webhook secret from the vault and extract the Slack webhook URL from it. | ||
| $slack_webhook_secret = "$(imp-ci secrets read --environment=production --buildkite-org=improbable --secret-type=slack-webhook --secret-name=unreal-gdk-slack-web-hook)" | ||
| $slack_webhook_url = $slack_webhook_secret | ConvertFrom-Json | %{$_.url} | ||
|
|
||
| $gdk_commit_url = "https://github.com/spatialos/UnrealGDK/commit/${gdk_commit_hash}" | ||
| $project_commit_url = "https://github.com/spatialos/UnrealGDKExampleProject/commit/$env:BUILDKITE_COMMIT" | ||
| $build_url = "$env:BUILDKITE_BUILD_URL" | ||
| $json_message = [ordered]@{ | ||
| text = "$slack_text" | ||
| attachments= @( | ||
| @{ | ||
| fallback = "Find build here: $build_url." | ||
| color = "good" | ||
| fields = @( | ||
| @{ | ||
| title = "Build Message" | ||
| value = "$env:BUILDKITE_MESSAGE".Substring(0, [System.Math]::Min(64, "$env:BUILDKITE_MESSAGE".Length)) | ||
| short = "true" | ||
| } | ||
| @{ | ||
| title = "Example Project branch" | ||
| value = "$env:BUILDKITE_BRANCH" | ||
| short = "true" | ||
| } | ||
| @{ | ||
| title = "GDK branch" | ||
| value = "$gdk_branch_name" | ||
| short = "true" | ||
| } | ||
| ) | ||
| actions = @( | ||
| @{ | ||
| type = "button" | ||
| text = ":github: Project commit" | ||
| url = "$project_commit_url" | ||
| style = "primary" | ||
| } | ||
| @{ | ||
| type = "button" | ||
| text = ":github: GDK commit" | ||
| url = "$gdk_commit_url" | ||
| style = "primary" | ||
| } | ||
| @{ | ||
| type = "button" | ||
| text = ":buildkite: BK build" | ||
| url = "$build_url" | ||
| style = "primary" | ||
| } | ||
| ) | ||
| } | ||
| ) | ||
| } | ||
|
|
||
| if ($launch_deployment -eq "true") { | ||
| for ($i = 0; $i -lt $engine_version_count; $i++){ | ||
| $deployment_name = buildkite-agent meta-data get "deployment-name-$($i+1)" | ||
| $deployment_url = "https://console.improbable.io/projects/${project_name}/deployments/${deployment_name}/overview" | ||
| $deployment_button = @{ | ||
| type = "button" | ||
| text = ":cloud: Deployment $($i+1)" | ||
|
There was a problem hiding this comment. Is there a way we can name the deployment after the engine version? Not essential. There was a problem hiding this comment. I tried to, but hit the deployment name limit, I guess we could just change the entire format so it no longer includes the full timestamp or the gdk commit or something, but I thought it could be annoying to do. There was a problem hiding this comment. Perhaps you could put it in the deployment description? I've never used it, but apparently there is a There was a problem hiding this comment. Seems good, I can try that There was a problem hiding this comment. Added this, but I can't find the description anywhere in the console :D |
||
| url = "$deployment_url" | ||
| style = "primary" | ||
| } | ||
| $json_message["attachments"][0]["actions"] += ($deployment_button) | ||
| } | ||
| } | ||
|
|
||
| $json_request = $json_message | ConvertTo-Json -Depth 10 | ||
|
|
||
| Invoke-WebRequest -UseBasicParsing "$slack_webhook_url" -ContentType "application/json" -Method POST -Body "$json_request" | ||
| Finish-Event "slack-notify" "slack-notify" | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.