From cfcbb23ca410c6d124afea52aa0ad913156f7b1f Mon Sep 17 00:00:00 2001 From: Gerasimos Chourdakis Date: Mon, 27 Oct 2025 22:59:36 +0100 Subject: [PATCH 1/5] Update and slim down vm/container slides for ws2526 --- .../containers_slides.md | 100 ------------------ .../docker_slides.md | 44 ++++---- .../singularity_slides.md | 2 +- .../vagrant_slides.md | 16 +-- .../virtualbox_slides.md | 65 ++++++++---- .../virtualmachines_slides.md | 100 ------------------ 6 files changed, 72 insertions(+), 255 deletions(-) delete mode 100644 02_virtualization_and_containers/containers_slides.md delete mode 100644 02_virtualization_and_containers/virtualmachines_slides.md diff --git a/02_virtualization_and_containers/containers_slides.md b/02_virtualization_and_containers/containers_slides.md deleted file mode 100644 index 3dda6593..00000000 --- a/02_virtualization_and_containers/containers_slides.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -type: slide -slideOptions: - transition: slide - width: 1400 - height: 900 - margin: 0.1 ---- - - - -# Containers - ---- - -## What is a Container? - - - -- Container operates in "fenced off" part of the operating system (`namespaces`) -- Lower overhead than virtual machines - - Runs on kernel (and libraries) of the host OS - - Cheap to start and stop a container -- Available features depend on Host (Linux, Windows) -- Container can be isolated. - ---- - -## Common Use-Cases - -- Microservices -- Reproducible environments for developing and testing (DevOps) -- Container hype strongly driven by [Docker](https://www.docker.com/). -- More and more in science - - High-performance computing, "Bring Your Own Environment" - - Reproducible research - ---- - -## Container Solutions - -- Plenty of different container formats - - [lxc/lxd](https://linuxcontainers.org/), [Docker](https://www.docker.com/), [Apptainer](https://apptainer.org/), [podman](https://podman.io/), [Sarus](https://user.cscs.ch/tools/containers/sarus/)... -- Different solutions with different strengths due to different use cases - - Working on the (Super-)Userspace - - Direct access to hardware vs. encapsulation - - Generic or with integration in software ecosystem (e.g. job schedulers) -- Effort to establish certain standards - - [Open Container Initiative (OCI)](https://opencontainers.org/) - ---- - -## Security - -- Containers are isolated - - > Applications are safer in containers and Docker provides the strongest default isolation capabilities in the industry - -- Read the manual -- Are third-part containers trust-worthy? - ---- - -## Summary - -- Shares many similarities with VMs, but - - Lightweight alternative to VMs - - Stricter limitations than VMs - - Often different use cases and working together -- Many different containers solutions - - Standardization effort - - Choose right solution for your use case - ---- - -## Further Reading - -- [Docker](https://www.docker.com/) -- [Docker Hub](https://hub.docker.com/) -- [Apptainer](https://apptainer.org/) -- [Sarus](https://user.cscs.ch/tools/containers/sarus/) -- [lxc/lxd](https://linuxcontainers.org/) -- [podman](https://podman.io/) -- [Open Container Initiative (OCI)](https://opencontainers.org/) -- [Singularity: Scientific containers for mobility of compute](https://doi.org/10.1371/journal.pone.0177459) -- [Malicious Docker Hub Container Images Used for Cryptocurrency Mining](https://www.trendmicro.com/vinfo/fr/security/news/virtualization-and-cloud/malicious-docker-hub-container-images-cryptocurrency-mining) diff --git a/02_virtualization_and_containers/docker_slides.md b/02_virtualization_and_containers/docker_slides.md index 9a269980..2c13e7d4 100644 --- a/02_virtualization_and_containers/docker_slides.md +++ b/02_virtualization_and_containers/docker_slides.md @@ -23,7 +23,7 @@ slideOptions: } -# Docker +# Containers with Docker @@ -31,38 +31,42 @@ slideOptions: --- -## What is Docker? 1/3 +## What is a Container? -- Docker Inc. -- Docker Desktop + - > Developer productivity tools and a local Kubernetes environment. - -- Docker Engine - - > an open source containerization technology for building and containerizing your applications. +- Container operates in "fenced off" part of the operating system (`namespaces`) +- Lower overhead than a virtual machine + - Runs on kernel (and libraries) of the host OS + - Cheap to start and stop a container +- Available features depend on Host (Linux, Windows) +- Container can be isolated. --- -## What is Docker? 2/3 - -- Docker Hub - - > Cloud-based application registry and development team collaboration services. - -- Docker Compose +## Common Use-Cases - > a tool for defining and running multi-container Docker applications. +- Microservices +- Reproducible environments for developing and testing (DevOps) +- More and more in science + - High-performance computing, "Bring Your Own Environment" + - Reproducible research --- -## What is Docker? 3/3 +## Container Solutions -> Docker is an open platform for developing, shipping, and running applications. +- Plenty of different container formats + - [lxc/lxd](https://linuxcontainers.org/), [Docker](https://www.docker.com/), [podman](https://podman.io/), [Apptainer](https://apptainer.org/), [Sarus](https://user.cscs.ch/tools/containers/sarus/)... +- Different solutions with different strengths due to different use cases + - Working on the (Super-)Userspace + - Direct access to hardware vs. encapsulation + - Generic or with integration in software ecosystem (e.g. job schedulers) +- Common standard: [Open Container Initiative (OCI)](https://opencontainers.org/) --- -## Introduction +## Docker - 2010: Docker Inc. founded - 2013: First Docker release diff --git a/02_virtualization_and_containers/singularity_slides.md b/02_virtualization_and_containers/singularity_slides.md index 8dd18b7c..0777d869 100644 --- a/02_virtualization_and_containers/singularity_slides.md +++ b/02_virtualization_and_containers/singularity_slides.md @@ -167,7 +167,7 @@ Details available in [`singularity_demo.md`](https://github.com/Simulation-Softw - Special container format for scientific computing - Immutable containers in a single file - Prefers integration over isolation -- Other solutions: [Sarus](https://user.cscs.ch/tools/containers/sarus/), [Charliecloud](https://hpc.github.io/charliecloud/) +- Other solutions: [Sarus](https://user.cscs.ch/tools/containers/sarus/), [Charliecloud](https://charliecloud.io/latest/index.html) --- diff --git a/02_virtualization_and_containers/vagrant_slides.md b/02_virtualization_and_containers/vagrant_slides.md index a62ecf6d..ec351ed3 100644 --- a/02_virtualization_and_containers/vagrant_slides.md +++ b/02_virtualization_and_containers/vagrant_slides.md @@ -55,13 +55,8 @@ slideOptions: ## What for? - Developing software - - Set up consistent development environment - - Share environment with others + - Set up / share consistent development environment - Testing software, workflows... - - Disposable environments - - Consistent workflows -- More general - - Simple way to setup virtual machine - Reproducible environment --- @@ -70,12 +65,11 @@ slideOptions: - Strong focus on workflow consistency - (Re)use existing images -- Automatize VM creation and configuration +- Automate VM creation and configuration - Easier than with VirtualBox CLI and shell scripts - Store in Git-friendly format -- For us: - - Management of VirtualBox VMs (testing, developing...) - - Sharing of VMs (debugging, workshops...) + +> "If you can't git diff a file format, it's broken." --- @@ -133,7 +127,7 @@ Details available in [`vagrant_demo.md`](https://github.com/Simulation-Software- ```ruby Vagrant.configure("2") do |config| config.vm.box = "bento/ubuntu-24.04" - config.vm.box_version = "202407.23.0" + config.vm.box_version = "202510.26.0" end ``` diff --git a/02_virtualization_and_containers/virtualbox_slides.md b/02_virtualization_and_containers/virtualbox_slides.md index 85c53ca6..63ae79e7 100644 --- a/02_virtualization_and_containers/virtualbox_slides.md +++ b/02_virtualization_and_containers/virtualbox_slides.md @@ -24,17 +24,49 @@ slideOptions: } -# VirtualBox +# Virtual machines with VirtualBox --- -## Introduction +## What is a Virtual Machine? + + + +- A computer inside your computer, with its own OS and resources +- Virtual Machines are portable and flexible + +--- + +## (Some) Features of Virtual Machines + +- Exclusive access to some of your resources +- Behaves like a native installation +- Isolation from host operating system +- Popular for + - Safety critical tasks + - Development and testing + - Wherever one wants a portable solution + +--- + +## Types of Hypervisors + +- **Type 1** + - Runs directly on bare-metal hardware, like the host OS + - Examples: [Microsoft Hyper-V](https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/about/), [VMware ESXi](https://www.vmware.com/products/esxi-and-esx.html), [Xen](https://xenproject.org/), [KVM](https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine), ... +- **Type 2** + - Negotiates resources shared with a host OS + - Examples: [VirtualBox](https://www.virtualbox.org/), [VMWare Workstation Player](https://www.vmware.com/products/workstation-player.html), [Parallels](https://www.parallels.com/eu/products/desktop/)... + +--- + +## VirtualBox - Hosted hypervisor created by Innotek GmbH (Weinstadt, Germany) - Obtained by Sun Microsystems in 2008. Since 2010, owned by Oracle -- Open source (GPLv3) +- Free software (GPLv3) --- @@ -50,14 +82,12 @@ slideOptions: --- -## Virtual Hard Drive Formats +## Note on CPU architectures -- Virtual hard drive as a file -- Multiple formats, for example: - - **VDI**: "Virtual Disk Image", VirtualBox' native format - - **VHD**: Format used by Microsoft - - **VMDK**: VMWare's virtual disk format - - Support for dynamic allocation +- VirtualBox runs both on x86 and ARM +- ARM hosts need ARM VMs (no emulation) +- Linux support limited/recent (e.g., Ubuntu 25.10) +- Emulation alternative (very slow): [UTM](https://mac.getutm.app/) --- @@ -89,21 +119,10 @@ Details available in [`virtualbox_demo.md`](https://github.com/Simulation-Softwa --- -## Summary and Outlook - -- VirtualBox is an open-source system for virtual machines -- Simple to set up and run -- Manual setup, but command-line interface exists -- Next step: Infrastructure as code - - Consistency of the environment? - - How to share a VM image with others? - - > "If you can't git diff a file format, it's broken." - ---- - ## Further Reading - [VirtualBox Manual](https://www.virtualbox.org/manual/UserManual.html) +- [VirtualBox Manual: 1. First Steps](https://www.virtualbox.org/manual/ch01.html) - [VirtualBox Manual: 4. Guest Additions](https://www.virtualbox.org/manual/ch04.html) +- [VirtualBox Manual: 13. Security Guide](https://www.virtualbox.org/manual/ch13.html) - [Overview of different disk formats](https://www.parallels.com/blogs/ras/vdi-vs-vhd-vs-vmdk/) diff --git a/02_virtualization_and_containers/virtualmachines_slides.md b/02_virtualization_and_containers/virtualmachines_slides.md deleted file mode 100644 index 8e39e6c3..00000000 --- a/02_virtualization_and_containers/virtualmachines_slides.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -type: slide -slideOptions: - transition: slide - width: 1400 - height: 900 - margin: 0.1 ---- - - - -# Virtual Machines - ---- - -## What is a Virtual Machine? - - - -- A computer inside your computer, with its own OS and resources -- Virtual Machines are portable and flexible - ---- - -## Common Terms - -- Host operating system - - The OS the hypervisor is installed on -- Guest operating system - - The OS running inside the virtual machine -- Virtual machine (VM) - - Environment the guest is running in - ---- - -## Types of Hypervisors - -- **Type 1** - - Runs directly on bare-metal hardware, like the host OS - - Examples: [Microsoft Hyper-V](https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/about/), [VMware ESXi](https://www.vmware.com/products/esxi-and-esx.html), [Xen](https://xenproject.org/), [KVM](https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine), ... -- **Type 2** - - Negotiates resources shared with a host OS - - Examples: [VirtualBox](https://www.virtualbox.org/), [VMWare Workstation Player](https://www.vmware.com/products/workstation-player.html), [Parallels](https://www.parallels.com/eu/products/desktop/)... - ---- - -## (Some) Features of Virtual Machines - -- Exclusive access to some of your resources -- Behaves like a native installation -- Isolation from host operating system -- Popular for - - Safety critical tasks - - Development and testing - - Wherever one wants a portable solution - ---- - -## Why is Virtualization Useful? - -- Running multiple operating systems simultaneously - - Test/develop/debug software for other OS -- Easier software installations and testing - - Preconfigured VMs for teaching (see [preCICE Demo VM](https://precice.org/installation-vm.html)) -- Testing and disaster recovery - - Create snapshots before testing, copy VMs etc. -- Separate services from each other for security (see [Qubes OS](https://www.qubes-os.org/intro/)) -- Reproducibility - ---- - -## Summary - -- Isolated environment that emulates a computer -- Different types of Hypervisors (bare-metal or running on OS) -- Run additional OSes on your machine -- VMs are portable, reproducible, easy to manage, and provide isolation - ---- - -### Further Reading - -- [VirtualBox Manual: 1. First Steps](https://www.virtualbox.org/manual/ch01.html) -- [VirtualBox Manual: 13. Security Guide](https://www.virtualbox.org/manual/ch13.html) -- ["How To Make Package Managers Cry"](https://archive.fosdem.org/2018/schedule/event/how_to_make_package_managers_cry/) - (Kenneth Hoste, FOSDEM 2018) From 3aefabc6ab06f91ecb23ffc8a6235b4df1c0919b Mon Sep 17 00:00:00 2001 From: Gerasimos Chourdakis Date: Tue, 28 Oct 2025 18:14:23 +0100 Subject: [PATCH 2/5] Update timetable --- timetable.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/timetable.md b/timetable.md index 3fe91fc5..d5b131c4 100644 --- a/timetable.md +++ b/timetable.md @@ -28,7 +28,12 @@ ## 3.1 – Wed, October 29, 2025 -(pending, slides update - VM and Containers) +- **5** min.: [Virtualization and Containers](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/intro_slides.md) +- **15** min.: VirtualBox: [slides](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualbox_slides.md), [demo](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualbox_demo.md) +- **20** min.: Vagrant: [slides](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/vagrant_slides.md), [demo](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/vagrant_demo.md) +- **30** min.: Docker: [slides](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_slides.md), [demo](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_demo.md) +- **20** min.: Singularity / Apptainer: [slides](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/singularity_slides.md), [demo](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/singularity_demo.md) + ## 3.2 – Wed, October 29, 2025 From 5e111aa3b11983d5b9c59d417c751d9e30a9983f Mon Sep 17 00:00:00 2001 From: Gerasimos Chourdakis Date: Tue, 28 Oct 2025 18:42:37 +0100 Subject: [PATCH 3/5] Update Singularity demo --- 02_virtualization_and_containers/singularity_demo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/02_virtualization_and_containers/singularity_demo.md b/02_virtualization_and_containers/singularity_demo.md index ef3603a9..1923f67b 100644 --- a/02_virtualization_and_containers/singularity_demo.md +++ b/02_virtualization_and_containers/singularity_demo.md @@ -19,7 +19,7 @@ ```Singularity BootStrap: docker - From: ubuntu:22.04 + From: precice/precice %post apt-get -y update @@ -30,7 +30,7 @@ export PATH=/usr/games:$PATH %runscript - date | cowsay | lolcat + precice-tools --version | cowsay | lolcat ``` - Show file `lolcow.def` content From 6ccd74c91ae8024a4a64c1444ffe25e748d6829e Mon Sep 17 00:00:00 2001 From: Gerasimos Chourdakis Date: Tue, 28 Oct 2025 18:56:48 +0100 Subject: [PATCH 4/5] Remove content from docker slides --- .../docker_slides.md | 16 ---------------- timetable.md | 1 - 2 files changed, 17 deletions(-) diff --git a/02_virtualization_and_containers/docker_slides.md b/02_virtualization_and_containers/docker_slides.md index 2c13e7d4..2415c0b3 100644 --- a/02_virtualization_and_containers/docker_slides.md +++ b/02_virtualization_and_containers/docker_slides.md @@ -77,16 +77,6 @@ slideOptions: --- -## Typical Docker Applications - -- Applications as Microservices -- Containers for consistent development environment -- Containers for consistent testing environment -- Portable format for sharing applications -- Avoid tedious installation procedures by providing Docker container ([FEniCS](https://fenicsproject.org/download/), [OpenFOAM](https://www.openfoam.com/download/openfoam-installation-on-windows-docker), [GitLab](https://docs.gitlab.com/ee/install/docker.html), etc.) - ---- - ## Building Blocks 1/2 - Docker objects @@ -248,12 +238,6 @@ Details available in [`docker_demo.md`](https://github.com/Simulation-Software-E --- -## Demo: Run FEniCS Container - -Details available in [`docker_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_demo.md) - ---- - ## Advanced Topics - User ID mapping diff --git a/timetable.md b/timetable.md index d5b131c4..aa82cbcf 100644 --- a/timetable.md +++ b/timetable.md @@ -34,7 +34,6 @@ - **30** min.: Docker: [slides](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_slides.md), [demo](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_demo.md) - **20** min.: Singularity / Apptainer: [slides](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/singularity_slides.md), [demo](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/singularity_demo.md) - ## 3.2 – Wed, October 29, 2025 - **90** min.: [Exercise: Virtualization and Containers](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualmachines_containers_exercise.md) From d4f76730b7280562ccce74abba295e1a9b173b72 Mon Sep 17 00:00:00 2001 From: Gerasimos Chourdakis Date: Tue, 28 Oct 2025 19:01:56 +0100 Subject: [PATCH 5/5] Fix Docker logo --- 02_virtualization_and_containers/docker_slides.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/02_virtualization_and_containers/docker_slides.md b/02_virtualization_and_containers/docker_slides.md index 2415c0b3..b4f9f4e4 100644 --- a/02_virtualization_and_containers/docker_slides.md +++ b/02_virtualization_and_containers/docker_slides.md @@ -25,7 +25,7 @@ slideOptions: # Containers with Docker - + Image by https://www.docker.com/, Fair use, https://en.wikipedia.org/w/index.php?curid=70663056