Skip to content

Commit

Permalink
Merge architecture_2024 branch into master (#1177)
Browse files Browse the repository at this point in the history 
After a few months of work, it is time to merge the `architecture_2024`
branch into `master`. It is still a work-in-progress, but all the
efforts should be go now against that branch.

## Pull requests

* #1061
* #1064
* #1073
* #1074
* #1080
* #1089
* #1091
* #1092
* #1094
* #1095
* #1099
* #1100
* #1102
* #1103
* #1112
* #1114
* #1116
* #1117
* #1119
* #1120
* #1123
* #1126
* #1129
* #1130
* #1131
* #1132
* #1133
* #1134
* #1136
* #1139
* #1140
* #1143
* #1146

## Other commits

* 8efa41f
* 9e2dec0
  • Loading branch information
@imobachgs
imobachgs committed May 6, 2024
2 parents aa89adb + 6fb21b6 commit 3353513
Show file tree
Hide file tree
Showing 192 changed files with 9,401 additions and 3,903 deletions.
70 changes: 35 additions & 35 deletions .github/workflows/ci-web.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,38 +57,38 @@ jobs:

- name: Install dependencies
run: npm install

- name: Build the application
run: make

- name: Run check spell
run: npm run cspell

- name: Check types
run: npm run check-types

- name: Run ESLint
run: npm run eslint

- name: Run Stylelint
run: npm run stylelint

- name: Run the tests and generate coverage report
run: npm test -- --coverage

# send the code coverage for the web part to the coveralls.io
- name: Coveralls GitHub Action
uses: coverallsapp/github-action@v2
with:
base-path: ./web
flag-name: web
parallel: true

# close the code coverage and inherit the previous coverage for the Ruby and
# Rust parts (it needs a separate step, the "carryforward" flag can be used
# only with the "parallel-finished: true" option)
- name: Coveralls Finished
uses: coverallsapp/github-action@v2
with:
parallel-finished: true
carryforward: "service,rust"
#
# - name: Build the application
# run: make
#
# - name: Run check spell
# run: npm run cspell
#
# - name: Check types
# run: npm run check-types
#
# - name: Run ESLint
# run: npm run eslint
#
# - name: Run Stylelint
# run: npm run stylelint
#
# - name: Run the tests and generate coverage report
# run: npm test -- --coverage
#
# # send the code coverage for the web part to the coveralls.io
# - name: Coveralls GitHub Action
# uses: coverallsapp/github-action@v2
# with:
# base-path: ./web
# flag-name: web
# parallel: true
#
# # close the code coverage and inherit the previous coverage for the Ruby and
# # Rust parts (it needs a separate step, the "carryforward" flag can be used
# # only with the "parallel-finished: true" option)
# - name: Coveralls Finished
# uses: coverallsapp/github-action@v2
# with:
# parallel-finished: true
# carryforward: "service,rust"
34 changes: 34 additions & 0 deletions doc/agama-security.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
## Agama Concepts

Agama's functionality is divided into backend and frontend. Communication between two parts is done through HTTP/JSON and/or websocket. Most of the api requires an authorization.

As frontend Agama offers a web based user interface (web UI) or a commandline interface (CLI). Backend currently is bunch of services implemented in Rust or Ruby with support from YaST libraries. For interprocess communication Agama uses D-Bus.

### Authorization

Authorization is done via password. To get authorized the frontend has to provide the root password (root on the backend's system). The password is validated through PAM [1]. Once the authorization succeeds, the backend generates an authorization token and passes it back to frontend. Agama uses [JSON Web Token (JWT)] [2] as authorization token [3]. All subsequent calls to the API has to be done together with the token. In case of the web UI, the token is stored in a HTTP-only cookie.

Agama supports special use case when Agama's UI or CLI is used in live installation media. In such case skipping autorization is supported to get feeling of using a desktop application. However, skipping authorization happens only for local access. When connecting remotely, authorization is still in place. Skipping of authorization is made possible thanks to option ```--generate-token```. When this option is used, Agama's web server service generates valid JWT automatically on start. The token is stored locally [4]. To make it usable for web UI, token is imported into web browser's internal database by Agama provided startup [5] script. The script prepares custom profile for Firefox with predefined homepage pointing to Agama's special login page with the generated token as part of a get request in the homepage url. As part of the response, the token is stored as `httpOnly` cookie. In case of CLI the situation is way easier as the token can be accessed and used directly as needed from well known location [4].

### JWT

The token carries just one claim - the expiration date. Token's lifetime is currently set to one day. The token is provided in encrypted form. Security key is either automatically created random string [6] which is 30 characters long. However, security can be provided via the `jwt_secret` option in the `/etc/agama.d/server.yaml` agama's configuration file. The content of this option is expected to be a string but no checks are done.

### Communication between the frontend and the backend

If both components run locally, communication can be done over HTTP or HTTPS. However, in case when both run on different machines, HTTPS is mandatory. In such case all HTTP requests are automatically redirected to HTTPS. A HTTP response with code 308 (permanent redirect) is returned in such case.

For notifications on changes from backend Agama uses WebSocket technology. Typically backend notificates about installation progress or network configuration changes this way.

### HTTPS certificates

SSL communication is secured either by self-signed certificate which is automatically generated by Agama if no certificate was provided by user. If Agama should use particular custom certificate Agama's web server provides options --cert and --key for path to certificate respectively to private key (in PEM format).

## Links to external sources

- [1] Rust PAM crate, https://crates.io/crates/pam
- [2] RFC 7519, http://jwt.io
- [3] Rust jsonwebtoken crate, https://crates.io/crates/jsonwebtoken
- [4] Backend's machine at /run/agama/token
- [5] [Firefox startup script] See https://github.com/openSUSE/agama/blob/master/live/root/root/.icewm/startup
- [6] Rust rand crate, https://crates.io/crates/rand
179 changes: 79 additions & 100 deletions live/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Live ISO

<!-- omit from toc -->

## Table of Content

- [Live ISO](#live-iso)
Expand All @@ -26,19 +27,15 @@

## Layout

This directory contains a set of files that are used to build the Agama Live ISO
image.
This directory contains a set of files that are used to build the Agama Live ISO image.

- [src](src) subdirectory contains all source files which are copied unmodified
to the OBS project
- [root](root) subdirectory contains files which are added to the Live ISO root
system (inside the squashfs image)
- [root-ALP-PXE](root-ALP-PXE) subdirectory contains specific files for the ALP
image used for the PXE boot, see a separate [PXE documentation](PXE.md) for
more details about the PXE boot
- [config-cdroot](config-cdroot) subdirectory contains file which are copied to
the uncompressed root of the ISO image, the files can be accessed just by
mounting the ISO file or the DVD medium
- [src](src) subdirectory contains all source files which are copied unmodified to the OBS project
- [root](root) subdirectory contains files which are added to the Live ISO root system (inside the
squashfs image)
- [root-ALP-PXE](root-ALP-PXE) subdirectory contains specific files for the ALP image used for the
PXE boot, see a separate [PXE documentation](PXE.md) for more details about the PXE boot
- [config-cdroot](config-cdroot) subdirectory contains file which are copied to the uncompressed
root of the ISO image, the files can be accessed just by mounting the ISO file or the DVD medium

## Building the Sources

Expand Down Expand Up @@ -66,31 +63,28 @@ To build the ISO locally run the
make build
```

command. The built ISO image is saved to the `/var/tmp/build-root` directory,
see the end of the build for output for the exact ISO file name.
command. The built ISO image is saved to the `/var/tmp/build-root` directory, see the end of the
build for output for the exact ISO file name.

For building an ISO image you need a lot of free space at the `/var` partition.
Make sure there is at least 25GiB free space otherwise the build will
fail.
For building an ISO image you need a lot of free space at the `/var` partition. Make sure there is
at least 25GiB free space otherwise the build will fail.

### Build Options

By default this will build the openSUSE image. If you want to build
another image then run
By default this will build the openSUSE image. If you want to build another image then run

```shell
make build FLAVOR=<flavor>
# for building the ALP flavor:
make build FLAVOR=ALP
```

See the [_multibuild](src/_multibuild) file for the list of available build
flavors.
See the [_multibuild](src/_multibuild) file for the list of available build flavors.

By default it will use the [systemsmanagement:Agama:Staging](
https://build.opensuse.org/project/show/systemsmanagement:Agama:Staging) OBS
project. If you want to build using another project, like your fork, then delete
the `dist` directory and checkout the OBS project manually and run the build:
By default it will use the
[systemsmanagement:Agama:Staging](https://build.opensuse.org/project/show/systemsmanagement:Agama:Staging)
OBS project. If you want to build using another project, like your fork, then delete the `dist`
directory and checkout the OBS project manually and run the build:

```shell
rm -rf dist
Expand All @@ -101,140 +95,125 @@ make build

## Image Definition

The [KIWI](https://github.com/OSInside/kiwi) image builder is used by OBS to
build the Live ISO. See the [KIWI documentation](
https://osinside.github.io/kiwi/index.html) for more details about the build
workflow and the `.kiwi` file format.
The [KIWI](https://github.com/OSInside/kiwi) image builder is used by OBS to build the Live ISO. See
the [KIWI documentation](https://osinside.github.io/kiwi/index.html) for more details about the
build workflow and the `.kiwi` file format.

### KIWI Files

The main Kiwi source files are located in the [src](src) subdirectory:

- [agama-live.kiwi](src/agama-live.kiwi) is the main KIWI file which drives the
ISO image build.
- [config.sh](src/config.sh) is a KIWI hook script which is called and the end
of the build process, after all packages are installed but before compressing
and building the image. The script runs in the image chroot and is usually
used to adjust the system configuration (enable/disable services, patching
configuration files or deleting not needed files).
- [_constraints](src/_constraints) file tells OBS to build the image on the
hosts with enough resources (enough free disk space).
- [_multibuild](src/_multibuild) defines the image flavors (KIWI profiles)
which are available to build
- [images.sh](src/images.sh) - injects a script which checks whether the machine
has enough RAM when booting the Live ISO
- [fix_bootconfig](src/fix_bootconfig) - a special KIWI hook script which sets
the boot configuration on S390 and PPC64 architectures.
- [agama-live.kiwi](src/agama-live.kiwi) is the main KIWI file which drives the ISO image build.
- [config.sh](src/config.sh) is a KIWI hook script which is called and the end of the build process,
after all packages are installed but before compressing and building the image. The script runs in
the image chroot and is usually used to adjust the system configuration (enable/disable services,
patching configuration files or deleting not needed files).
- [_constraints](src/_constraints) file tells OBS to build the image on the hosts with enough
resources (enough free disk space).
- [_multibuild](src/_multibuild) defines the image flavors (KIWI profiles) which are available to
build
- [images.sh](src/images.sh) - injects a script which checks whether the machine has enough RAM when
booting the Live ISO
- [fix_bootconfig](src/fix_bootconfig) - a special KIWI hook script which sets the boot
configuration on S390 and PPC64 architectures.

## Image Configuration

The Live ISO is configured to allow using some features and allow running Agama
there.
The Live ISO is configured to allow using some features and allow running Agama there.

### SSH Server

The SSH connection for the root user is enabled in the [10_root_login.conf](
root/etc/ssh/sshd_config.d/10_root_login.conf) file.
The SSH connection for the root user is enabled in the
[10_root_login.conf](root/etc/ssh/sshd_config.d/10_root_login.conf) file.

### Autologin

Automatic root login and staring the graphical environment is configured in
several files.
Automatic root login and staring the graphical environment is configured in several files.

- [x11-autologin.service](src/etc/systemd/system/x11-autologin.service) uses
`startx` to start an x11 session.
- `startx` runs the Icewm window manager via [.xinitrc](root/root/.xinitrc)
file.
- [x11-autologin.service](src/etc/systemd/system/x11-autologin.service) uses `startx` to start an
x11 session.
- `startx` runs the Icewm window manager via [.xinitrc](root/root/.xinitrc) file.
- Icewm autostarts Firefox via [startup](root/root/.icewm/startup) file.
- Icewm uses the usual YaST2 installation
[preferences.yast2](root/etc/icewm/preferences.yast2) configuration file
- Icewm uses the usual YaST2 installation [preferences.yast2](root/etc/icewm/preferences.yast2)
configuration file

### Firefox Profile

The default Firefox configuration is defined in the
[profile](root/root/.mozilla/firefox/profile) file. It disables several features
which do not make sense in Live ISO like remembering the used passwords.
The default Firefox configuration is defined in the [profile](root/root/.mozilla/firefox/profile)
file. It disables several features which do not make sense in Live ISO like remembering the used
passwords.

### Dracut menu

The [98dracut-menu](live/root/usr/lib/dracut/modules.d/98dracut-menu) directory
implements a simple menu system for dracut. To activate it
during boot add `rd.cmdline=menu` to the boot prompt. This is similar to
`rd.cmdline=ask` which gives you a simple one-line prompt to add boot options.
The [98dracut-menu](live/root/usr/lib/dracut/modules.d/98dracut-menu) directory implements a simple
menu system for dracut. To activate it during boot add `rd.cmdline=menu` to the boot prompt. This is
similar to `rd.cmdline=ask` which gives you a simple one-line prompt to add boot options.

The dracut-cmdline-menu can currently set the `root` and `proxy` options. The
settings are copied (using a dracut pre-pivot hook) to the live system in
The dracut-cmdline-menu can currently set the `root` and `proxy` options. The settings are copied
(using a dracut pre-pivot hook) to the live system in
[cmdline-menu.conf](root/etc/cmdline-menu.conf).

There is also the complete command line in the
[cmdline-full.conf](root/etc/cmdline-full.conf) file - maybe it can useful at
least for debugging.
There is also the complete command line in the [cmdline-full.conf](root/etc/cmdline-full.conf)
file - maybe it can useful at least for debugging.

For more details see [dracut.bootup(7)](
https://man.archlinux.org/man/dracut.bootup.7.en),
[dracut-pre-pivot.service(8)](
https://man.archlinux.org/man/extra/dracut/dracut-pre-pivot.service.8.en).
For more details see [dracut.bootup(7)](https://man.archlinux.org/man/dracut.bootup.7.en),
[dracut-pre-pivot.service(8)](https://man.archlinux.org/man/extra/dracut/dracut-pre-pivot.service.8.en).

To arrange the dracut config in KIWI you have to adjust the default dracut
config of the live system. This is done in [config.sh](src/config.sh). You can
also fill in a default network location if one is defined for a product
(currently not).
To arrange the dracut config in KIWI you have to adjust the default dracut config of the live
system. This is done in [config.sh](src/config.sh). You can also fill in a default network location
if one is defined for a product (currently not).

### Avahi/mDNS

The mDNS service allows resolving host names in the local network without
a DNS server. That is implemented by the `avahi-daemon` service which enabled
in the [config.sh](src/config.sh) file and installed in the `avahi` RPM package.
The mDNS service allows resolving host names in the local network without a DNS server. That is
implemented by the `avahi-daemon` service which enabled in the [config.sh](src/config.sh) file and
installed in the `avahi` RPM package.

The mDNS protocol resolves the hosts in the `.local` domain.

#### The Default Hostname

By default the Agama live ISO sets the `agama` host name which can be used
as `agama.local` full hostname in URL.
By default the Agama live ISO sets the `agama` host name which can be used as `agama.local` full
hostname in URL.

The default hostname is set by the
[agama-hostname](root/etc/systemd/system/agama-hostname.service) service.
The default hostname is set by the [agama-hostname](root/etc/systemd/system/agama-hostname.service)
service.

If the hostname is set via the `hostname=` boot parameter then the `agama`
host name is not used, the boot option takes precedence.
If the hostname is set via the `hostname=` boot parameter then the `agama` host name is not used,
the boot option takes precedence.

#### Service Advertisement

The Avahi HTTPS service announcement is configured via the Avahi
[agama.service](root/etc/avahi/services/agama.service) file

That allows scanning all running Agama instances in the local network with
command:
That allows scanning all running Agama instances in the local network with command:

```shell
avahi-browse -t -r _agama._sub._https._tcp
```

### The Default Cockpit/Agama TCP Port

The default Cockpit TCP port is 9090. That makes sense for the system management
framework as the default ports might be used by a running Apache or other web
servers.
The default Cockpit TCP port is 9090. That makes sense for the system management framework as the
default ports might be used by a running Apache or other web servers.

But Agama runs from a Live ISO where running a web server does not make much
sense so we can safely use the default HTTP(S) ports.
But Agama runs from a Live ISO where running a web server does not make much sense so we can safely
use the default HTTP(S) ports.

The default port is changed in the
[listen.conf](root/etc/systemd/system/cockpit.socket.d/listen.conf) file.

### Autoinstallation Support

The autoinstallation is started using the
[agama-auto](root/etc/systemd/system/agama-auto.service) service which starts
the [auto.sh](root/usr/bin/auto.sh) script. This script downloads the
The autoinstallation is started using the [agama-auto](root/etc/systemd/system/agama-auto.service)
service which starts the [auto.sh](root/usr/bin/auto.sh) script. This script downloads the
installation profile, applies it to Agama and starts the installation.

### Firmware Cleanup

The [fw_cleanup.rb](root/tmp/fw_cleanup.rb) script removes the unused firmware
from the image. Many firmware files are not needed, this makes the final ISO
much smaller.
The [fw_cleanup.rb](root/tmp/fw_cleanup.rb) script removes the unused firmware from the image. Many
firmware files are not needed, this makes the final ISO much smaller.

This script is started from [config.sh](src/config.sh) the script and after
running it the script deleted. (Not needed anymore in the system.)
This script is started from [config.sh](src/config.sh) the script and after running it the script
deleted. (Not needed anymore in the system.)

0 comments on commit 3353513

Please sign in to comment.