de.NBI integration process

[inesouledelhaj]

🧩 de.NBI Cloud Integration Process

This discussion documents the integration of the de.NBI Cloud into the Jupyter4NFDI infrastructure.

---

1. Overview

Resource provider: de.NBI Cloud
Goal: Integrate de.NBI Cloud compute resources into the Jupyter4NFDI infrastructure.
Target users: Researchers within NFDI community.
Resource hosting sites: Bielefeld, Berlin, Freiburg, Gießen, Heidelberg, Tübingen.
Responsible admins / contacts: Tim Kreuzer and Ines Ouled Elhaj.
Status: ✅ Integration completed.

---

2. Integration Tasks & Progress

Task	Status
Setup Infrastructure	✅ Completed
Install cluster requirements	✅ Completed
Install Outpost	✅ Completed
Configure Offered Options	✅ Completed
Connect Outpost to Staging Instance	✅ Completed
User Storage Integration	✅ Completed
Move to Production / Replicate Setup	✅ Completed

---

3. Task Details

Setup Infrastructure

Created network and security group then created the first main node of the RKE2 kubernetes cluster and initiated the cluster with the cloud init file. Added nodes using a script.

Install cluster requirements

• Install fleet manually to automate software updates.

• Created a dedicated branch containing all GitRepos for the required software components.

• For each software package, ccreated a Fleet configuration branch containing the corresponding fleet.yaml file (ingress-nginx, cert-manager, database-operator, openstack-cinder-csi).

• Manually created a Fleet GitRepo to automate the installation of all the required software GitRepos.

Install Outpost

Deploy and configure the JupyterHub Outpost component using Helm. Installation guidelines can be found in the documentation.

Configure Offered Options

Configured the system to offer multiple options: custom docker images, repo2docker and a minimal jupyterlab setup.
The system also offers multiple flavors:

2GB RAM, 1VCPU, 2 hours
2GB RAM, 1VCPU, 5 days
4GB RAM, 1VCPUs, 2 days
8GB RAM, 2VCPUs, 10 hours

Connect to Staging

To connect the Outpost to the Jupyter4NFDI staging instance, key information was exchanged between the outpost and the hub:

• The ssh-publickey was provided by the hub administrator.
• The hub administrator needed:
  • The IP address and port where to reach Outpost HTTP API
  • The IP address and port to reach the outpost ssh daemon

Connectivity to the staging JupyterHub instance was tested successfully.

User Storage Integration

Attached a persistent volume to one main node and set up an NFS server to provide shared user storage across all nodes.
Installed Storage Manager to manage user quotas via an XFS volume with xfs_quota, ensuring each user is limited to 30 GB of storage.

Move to Production

Replicated to production Jupyter4NFDI instance after testing the whole setup.