Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: eclipse-theia/theia-ide
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: eclipsesource/theia-ide
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Able to merge. These branches can be automatically merged.
  • 1 commit
  • 12 files changed
  • 1 contributor

Commits on Mar 17, 2025

  1. Theia Cloud Preview Deployments 

    * Build Theia IDE Preview Image
* Disable Licence check on fork
* Push to Artifact Registry
* Create Cluster Infrastructure
* Deploy App Definition
* Cleanup App Definition
* install missing dependencies on ubuntu system
* allow multiple sessions per user
    @jfaltermeier
    jfaltermeier committed Mar 17, 2025

    Verified

    This commit was signed with the committer’s verified signature.
    jfaltermeier Johannes Faltermeier
    GPG key ID: D5A93287C007D3BD
    Verified
    Learn about vigilant mode
    Copy the full SHA
    61f0d46 View commit details
6 changes: 3 additions & 3 deletions .github/workflows/license-check-workflow.yml
View file
Original file line number Diff line number Diff line change
@@ -5,9 +5,9 @@ on:
branches:
- master
workflow_dispatch:
pull_request:
branches:
- master
# pull_request:
# branches:
# - master
schedule:
- cron: '0 4 * * *' # Runs every day at 4am: https://docs.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule

111 changes: 111 additions & 0 deletions .github/workflows/publish-theia-ide-preview-cleanup.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Cleanup Theia IDE Preview Deployment

permissions:
contents: read
pull-requests: write

on:
workflow_dispatch:
inputs:
tag:
description: The image's tag
required: true
default: next
theia_version:
description: Theia Version
required: false
default: next

pull_request:
branches: [master]
types:
- closed

jobs:
build:
name: Cleanup Theia IDE Preview image
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set the Theia version
id: set-theia-version
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "THEIA_VERSION=${{ github.event.inputs.theia_version }}" >> $GITHUB_ENV
elif [ "${{ github.event_name }}" == "pull_request" ]; then
echo "THEIA_VERSION=" >> $GITHUB_ENV
else
echo "THEIA_VERSION=next" >> $GITHUB_ENV
fi
- name: Set image tag
id: set-tag
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "IMAGE_TAG=${{ github.event.inputs.tag }}" >> $GITHUB_ENV
elif [ "${{ github.event_name }}" == "pull_request" ]; then
echo "IMAGE_TAG=pr-${{ github.event.pull_request.number }}" >> $GITHUB_ENV
elif [ "${{ github.event_name }}" == "schedule" ]; then
echo "IMAGE_TAG=weekly" >> $GITHUB_ENV
else
echo "IMAGE_TAG=other" >> $GITHUB_ENV
fi
- name: Authenticate to Google Cloud
uses: google-github-actions/auth@71f986410dfbc7added4569d411d040a91dc6935 # v2.1.8
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
project_id: kubernetes-238012
create_credentials_file: true

- name: Setup Google Cloud SDK
uses: google-github-actions/setup-gcloud@77e7a554d41e2ee56fc945c52dfd3f33d12def9a # v2.1.4

- name: Get GKE Credentials
uses: google-github-actions/get-gke-credentials@d0cee45012069b163a631894b98904a9e6723729 # v2.3.3
with:
cluster_name: github-theia-ide-preview
location: europe-west3-c

- name: List sessions in theia-cloud namespace
run: kubectl get sessions -n theia-cloud

- name: List apps in theia-cloud namespace
run: kubectl get appdefinitions -n theia-cloud

- name: Delete app definition if existent
run: kubectl delete appdefinitions theia-ide-${{ env.IMAGE_TAG }} -n theia-cloud || true

- name: Delete existing sessions
run: kubectl get sessions -n theia-cloud -o json | jq -r '.items[] | select(.spec.appDefinition == "theia-ide-${{ env.IMAGE_TAG }}") | .metadata.name' | xargs -r kubectl delete sessions -n theia-cloud

- name: Update bot comment
if: github.event_name == 'pull_request'
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #7.0.1
with:
script: |
const {data: comments} = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.number,
})
const botComment = comments.find(comment => comment.user.id === 41898282)
const commentBody = "No Preview Deployment"
if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: commentBody
})
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.number,
body: commentBody
})
}
230 changes: 230 additions & 0 deletions .github/workflows/publish-theia-ide-preview-deployment.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
name: Publish Theia IDE Preview Deployment

permissions:
contents: read
pull-requests: write

on:
workflow_dispatch:
inputs:
tag:
description: The image's tag
required: true
default: next
theia_version:
description: Theia Version
required: false
default: next

pull_request:
branches: [master]
types:
- opened
- synchronize
- reopened

schedule:
- cron: '0 13 * * 0'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
build:
name: Build Theia IDE Preview image
runs-on: ubuntu-latest

steps:
# Check out the repository
- name: Checkout
uses: actions/checkout@v2

# Set up Node.js environment
- name: Use Node.js 20.x
uses: actions/setup-node@v3
with:
node-version: 20.x
registry-url: 'https://registry.npmjs.org'

# Set up Python environment
- name: Use Python 3.11
uses: actions/setup-python@v4
with:
python-version: '3.11'

# Install system dependencies required for native-keymap
- name: Install dependencies for native-keymap
run: sudo apt-get update && sudo apt-get install -y libx11-dev libxkbfile-dev

# Set up Docker Buildx for building Docker images
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349 # v3.7.1

# Determine the Theia version to update to. May be empty
- name: Set the Theia version
id: set-theia-version
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "THEIA_VERSION=${{ github.event.inputs.theia_version }}" >> $GITHUB_ENV
elif [ "${{ github.event_name }}" == "pull_request" ]; then
echo "THEIA_VERSION=" >> $GITHUB_ENV
else
echo "THEIA_VERSION=next" >> $GITHUB_ENV
fi
# Run the Theia update process if a specific version is set
- name: Run Theia update if version is specified
if: ${{ env.THEIA_VERSION != '' }}
run: |
yarn && yarn update:theia ${THEIA_VERSION} && yarn update:theia:children ${THEIA_VERSION} && yarn
# Determine the image tag to use for Docker based on the event
- name: Set image tag
id: set-tag
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "IMAGE_TAG=${{ github.event.inputs.tag }}" >> $GITHUB_ENV
elif [ "${{ github.event_name }}" == "pull_request" ]; then
echo "IMAGE_TAG=pr-${{ github.event.pull_request.number }}" >> $GITHUB_ENV
elif [ "${{ github.event_name }}" == "schedule" ]; then
echo "IMAGE_TAG=weekly" >> $GITHUB_ENV
else
echo "IMAGE_TAG=other" >> $GITHUB_ENV
fi
# Authenticate to Google Cloud using a service account
- name: Authenticate to Google Cloud
uses: google-github-actions/auth@71f986410dfbc7added4569d411d040a91dc6935 # v2.1.8
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
project_id: kubernetes-238012
create_credentials_file: true

# Set up Google Cloud SDK
- name: Setup Google Cloud SDK
uses: google-github-actions/setup-gcloud@77e7a554d41e2ee56fc945c52dfd3f33d12def9a # v2.1.4

# Configure Docker to use Google's Artifact Registry
- name: Configure Docker for Artifact Registry
run: |
gcloud auth configure-docker europe-west3-docker.pkg.dev
# Update or create a bot comment on pull requests
- name: Update bot comment
if: github.event_name == 'pull_request'
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #7.0.1
with:
script: |
const {data: comments} = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.number,
})
const botComment = comments.find(comment => comment.user.id === 41898282)
const commentBody = "Thank you for opening the PR!\n\nThis comment will be replaced with a link to a Preview deployment as soon as it is ready."
if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: commentBody
})
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.number,
body: commentBody
})
}
# Build and push the Docker image
- name: Build Docker image
uses: docker/build-push-action@48aba3b46d1b1fec4febb7c5d0c644b249a11355 # v6.10.0
with:
context: .
file: browser.Dockerfile
push: true
tags: |
europe-west3-docker.pkg.dev/kubernetes-238012/theia-ide-preview/theia-ide-preview:${{ env.IMAGE_TAG }}
# Get credentials for the Google Kubernetes Engine cluster
- name: Get GKE Credentials
uses: google-github-actions/get-gke-credentials@d0cee45012069b163a631894b98904a9e6723729 # v2.3.3
with:
cluster_name: github-theia-ide-preview
location: europe-west3-c

# List sessions running in the Theia Cloud namespace
- name: List sessions in theia-cloud namespace
run: kubectl get sessions -n theia-cloud

# List app definitions running in the Theia Cloud namespace
- name: List apps in theia-cloud namespace
run: kubectl get appdefinitions -n theia-cloud

# Delete existing app definition if it exists
- name: Delete app definition if existent
run: kubectl delete appdefinitions theia-ide-${{ env.IMAGE_TAG }} -n theia-cloud || true

# Delete existing sessions related to the specific app definition
- name: Delete existing sessions
run: kubectl get sessions -n theia-cloud -o json | jq -r '.items[] | select(.spec.appDefinition == "theia-ide-${{ env.IMAGE_TAG }}") | .metadata.name' | xargs -r kubectl delete sessions -n theia-cloud

# Create a new app definition for the deployment
- name: Create app definition
run: |
cat <<EOF | kubectl apply -f -
apiVersion: theia.cloud/v1beta10
kind: AppDefinition
metadata:
name: theia-ide-${{ env.IMAGE_TAG }}
namespace: theia-cloud
spec:
downlinkLimit: 30000
image: europe-west3-docker.pkg.dev/kubernetes-238012/theia-ide-preview/theia-ide-preview:${{ env.IMAGE_TAG }}
imagePullPolicy: Always
ingressname: theia-cloud-demo-ws-ingress
limitsCpu: "2"
limitsMemory: 500M
maxInstances: 3
minInstances: 0
mountPath: /home/project/persisted
name: theia-ide-${{ env.IMAGE_TAG }}
port: 3000
requestsCpu: "100m"
requestsMemory: 300M
timeout: 15
uid: 101
uplinkLimit: 30000
EOF
# Update the bot comment in pull requests with a link to the deployment
- name: Update bot comment with URL
if: github.event_name == 'pull_request'
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea #7.0.1
with:
script: |
const {data: comments} = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.number,
})
const botComment = comments.find(comment => comment.user.id === 41898282)
const commentBody = "Preview deployment created at https://launch.theia-ide-preview.eclipsesource-munich.com/?appDef=theia-ide-${{ env.IMAGE_TAG }}\n\nWhen the deployment is cleaned up, this link will be removed again."
if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: commentBody
})
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.number,
body: commentBody
})
}
3 changes: 3 additions & 0 deletions .terraform/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.terraform
terraform.tfstate
terraform.tfstate.backup
107 changes: 107 additions & 0 deletions .terraform/.terraform.lock.hcl
View file
56 changes: 56 additions & 0 deletions .terraform/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Terraform

## Overview

- **main.tf**: This central Terraform configuration file contains variable definitions, and infrastructure components such as storage buckets, clusters, node pools, Helm installations, and Kubernetes resources required for the Theia IDE Preview powered by Theia Cloud.

- **backend.tf**: Configures the Terraform backend to use Google Cloud Storage for state storage, defined within a specific bucket. This is a remote storage so that the state is stored at a central location and shared between all terraform users.

- **versions.tf**: This file specifies the required providers, and provider versions for the Terraform configuration.

- **.terraform.lock.hcl**: Automatically generated by Terraform to lock the provider versions, ensuring consistent and reproducible Terraform operations.

- **outputs.tf**: This Terraform configuration file defines output variables for resources, such as the IP address for nginx and URLs for keycloak and launch pages, allowing easy lookup.

- **keycloak.yaml**: Helm values for the keycloak installation that are not secrets passed to the Terraform chart.

- **theia-cloud.yaml**: Helm values for the Theia Cloud installation that are not secrets passed to the Terraform chart.

## main.tf

1. Creates a Kubernetes Cluster
2. Reserves a public IP
3. Installs Cert-Manager
4. Installs NginX Ingress Controller
5. Installs Theia Cloud Base and CRDs
6. Installs Keycloak
7. Configures the Keycloak Realm, OpenId Client, Github OAuth Provider, and all other required Keycloak related settings
8. Installs Theia Cloud
9. Configures Service Accounts to use this cluster from the Github Workflow

## Commands to use the chart

```sh
## Prerequisite: Authenticate with Google Cloud

## change to terraform directory
cd .terraform

## initial setup / sync state with remote location
GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token) terraform init

## apply changes
GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token) terraform apply

# Troubleshooting

## Unlock remote storage
GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token) terraform force-unlock <ID of the Lock Info>

## Import Browser Flow (find out id by changing something on this flow and checking the network tab in browser)
## https://theia-ide-preview.eclipsesource-munich.com/keycloak/admin/master/console/#/TheiaCloud/authentication -> browser flow -> change e.g. identity provider redirector -> inspect network
GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token) terraform import keycloak_authentication_execution.browser TheiaCloud/browser/0aac893f-210e-4183-a649-b70753868ede

## Inspect browser flow to fill out state in main.tf
GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token) terraform state show keycloak_authentication_execution.browser
```
5 changes: 5 additions & 0 deletions .terraform/backend.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
terraform {
backend "gcs" {
bucket = "github-theia-ide-preview-terraform-remote-backend"
}
}
26 changes: 26 additions & 0 deletions .terraform/keycloak.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
fullnameOverride: "keycloak"
httpRelativePath: "/keycloak/"

auth:
adminUser: admin

ingress:
enabled: true
ingressClassName: "nginx"
path: "/keycloak/"
annotations:
acme.cert-manager.io/http01-edit-in-place: "true"
nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
cert-manager.io/common-name: "theia-ide-preview.eclipsesource-munich.com"
tls: true
hostname: "theia-ide-preview.eclipsesource-munich.com"

postgresql:
enabled: true
postgresql_storageClass: ""
volumePermissions:
enabled: false

service:
type: "LoadBalancer"
494 changes: 494 additions & 0 deletions .terraform/main.tf
View file

Large diffs are not rendered by default.

11 changes: 11 additions & 0 deletions .terraform/outputs.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
output "nginx_ip" {
value = google_compute_address.host_ip.address
}

output "keycloak" {
value = "https://theia-ide-preview.eclipsesource-munich.com/keycloak"
}

output "launch_page" {
value = "https://launch.theia-ide-preview.eclipsesource-munich.com"
}
37 changes: 37 additions & 0 deletions .terraform/theia-cloud.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
app:
id: idepreview
name: Theia IDE Preview

demoApplication:
install: true # at the moment we will lose the ingress if we delete the last app def

hosts:
usePaths: false
configuration:
baseHost: theia-ide-preview.eclipsesource-munich.com
service: servicex
landing: launch
instance: a

landingPage:
ephemeralStorage: true

keycloak:
enable: true
realm: "TheiaCloud"
clientId: "theia-cloud"
authUrl: "https://theia-ide-preview.eclipsesource-munich.com/keycloak/"

operator:
eagerStart: false
bandwidthLimiter: "WONDERSHAPER"
sessionsPerUser: "3"
storageClassName: ""

ingress:
clusterIssuer: letsencrypt-prod
instances:
name: "theia-cloud-demo-ws-ingress"

monitor:
enable: false
12 changes: 12 additions & 0 deletions .terraform/versions.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
terraform {
required_providers {
kubectl = {
source = "gavinbunney/kubectl"
version = ">= 1.19.0"
}
keycloak = {
source = "mrparkers/keycloak"
version = ">= 4.4.0"
}
}
}