Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Rework of the runtime docs #9

Merged
merged 2 commits into from
Apr 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/execution/persistent.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

## Introduction

The Aleph.im network can run programs in two different manners:
The aleph.im network can run programs in two different manners:

* [on-demand execution](on_demand.md) runs programs only when needed, saving on resources. This is great to run programs
that are responding to user requests or API calls and can shutdown after processing the event.
* __persistent execution__ runs programs continuously. These are always running and great to run programs that cannot
afford to stop or need to handle incoming connections such as polling data from a websocket or AMQP API.

When a program is created with persistent execution enabled, the Aleph.im scheduler will find a Compute Resource Node
When a program is created with persistent execution enabled, the aleph.im scheduler will find a Compute Resource Node
(CRN) with enough resources to run the program and schedule the program to start on that node.

Persistent programs are designed to always run exactly once, and the scheduler will reallocate the program on another
Expand Down Expand Up @@ -75,7 +75,7 @@ uvicorn main:app --reload

## Step 2: Run a program in a persistent manner

To run the program in a persistent manner on the Aleph.im network, use:
To run the program in a persistent manner on the aleph.im network, use:

```shell
aleph program upload --persistent ./src/ main:app
Expand All @@ -93,4 +93,4 @@ TODO: Locate the CRN where your program is running.

## Conclusion

In this tutorial, you learned how to create and deploy persistent Virtual Machines on the Aleph.im network. You should now have a better understanding of how to use Aleph.im for distributed computing.
In this tutorial, you learned how to create and deploy persistent Virtual Machines on the aleph.im network. You should now have a better understanding of how to use aleph.im for distributed computing.
28 changes: 28 additions & 0 deletions docs/execution/runtimes/existing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Existing Runtimes

### Official runtime with Debian 11, Python 3.9 and NodeJS 14

Aleph.im provides users with a default runtime based on [Debian 11 "bullseye"](https://wiki.debian.org/DebianBullseye),
the current stable version of the Debian project which can
be [found on the Explorer](https://explorer.aleph.im/address/ETH/0x101d8D16372dBf5f1614adaE95Ee5CCE61998Fc9/message/STORE/bd79839bf96e595a06da5ac0b6ba51dea6f7e2591bb913deccded04d831d29f4).
This runtime is built with software available in the distribution, including Python 3.9 and Nodejs 14.

To optimize performance, this runtime uses a custom [Linux init](https://en.wikipedia.org/wiki/Init) process. This
process is specially designed to quickly launch the right endpoint in response to events such as HTTP requests. This is
especially useful when using [on-demand execution](../on_demand.md).

[//]: # (Not available yet)

[//]: # (### Official minimal runtime for binaries (Rust, Go, ...))

[//]: # ()

[//]: # (This official minimal runtime is designed to run Linux binaries quickly and efficiently. It is built on a minimal)

[//]: # (system, and does not include interpreters or virtual machines for popular programming languages. This makes launching)

[//]: # (binaries fast and efficient.)

### Third-party runtimes

Use third-party runtimes available on the network by specifying their `item_hash` when creating your program.
28 changes: 28 additions & 0 deletions docs/execution/runtimes/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Introduction

A program runtime is an operating system and software stack that enables your program to run on the aleph.im network.

Runtimes are customized Linux root filesystems that integrate with the aleph.im infrastructure and provide access to
APIs, as well as quick responses to HTTP requests and other events.

The project provides official runtimes with all you need for most programs. Additionally, you can build and publish
custom runtimes, and use any available runtime on the network for your program.

- [Use existing runtimes](./existing.md)
- [Create custom runtimes](./custom.md)


## Init process

[On-demand Execution](../on_demand.md) relies on a custom [Linux init](https://en.wikipedia.org/wiki/Init) process,
optimized to launch the right endpoint in response to events such as HTTP requests. This custom init consists in two
simple programs,
[init0.sh](https://raw.githubusercontent.com/aleph-im/aleph-vm/main/runtimes/aleph-alpine-3.13-python/init0.sh)
and [init1.py](https://raw.githubusercontent.com/aleph-im/aleph-vm/main/runtimes/aleph-alpine-3.13-python/init1.py).

Use these in your custom runtime by copying them to `/rootfs/sbin/init` and
`/mnt/rootfs/root/init1.py` respectively.

[Persistent Execution](../persistent.md) may use the same init process, but this is not required. If you do not make use
of the capabilities provided by the aleph.im ecosystem, using the default of your distribution
(ex: [systemd](https://systemd.io/), [OpenRC](https://github.com/OpenRC/openrc), ...) should work as well.
70 changes: 70 additions & 0 deletions docs/execution/runtimes/rebuild.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Custom Runtimes

Custom runtimes can be created by customizing the filesystem of Linux distributions or
[entirely from scratch](https://linuxfromscratch.org/lfs/).

## Base filesystem

Refer to the installation guide of your distribution for the procedure to obtain a base root filesystem.

> ⚠️ Only 64-bit Intel/AMD processors (x86_64) are supported on the network at the moment . Make sure you build runtimes on that architecture.

### Debian Bullseye

```shell
mkdir ./rootfs
debootstrap --arch=amd64 --variant=minbase bullseye ./rootfs http://deb.debian.org/debian/
```

### Ubuntu 22.04

```shell
mkdir ./rootfs
debootstrap --arch=amd64 jammy ./rootfs http://archive.ubuntu.com/ubuntu/
```

[//]: # (#### NixOS - TODO)


Following the next steps, you will be able to create and customize your own Debian based runtime:

## 1. Install required dependencies

First you need to install the required tools to be able to generate you own runtime.

Using Debian and Ubuntu systems:

```shell
sudo apt-get install debootstrap chroot mksquashfs
```

Using Nix:
```shell
nix-shell -p debootstrap squashfsTools
```

## 2. Obtain the build scripts

Download the files `create_disk_image.sh`, `init0.sh` and `init1.py` from the
[official runtime](existing.md#official-runtime-with-debian-11-python-39-and-nodejs-14):

[https://github.com/aleph-im/aleph-vm/tree/main/runtimes/aleph-debian-11-python](https://github.com/aleph-im/aleph-vm/tree/main/runtimes/aleph-debian-11-python)

Customize these files to suit your needs.

## 3. Build a new runtime

Running the script `create_disk_image.sh` as root will create a file named `rootfs.squashfs`:

```shell
sudo ./create_disk_image.sh
```

## 4. Publish the runtime on aleph.im

```shell
aleph file pin ./rootfs.squashfs
```

This command will provide you with the `item_hash` of the custom runtime,
that you can then use when creating the program.
4 changes: 2 additions & 2 deletions docs/execution/volumes/immutable.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Additional data can be provided to a program by bundling it in an immutable volu
mksquashfs ./custom-data data.squashfs
```

### 2. Publish the file on Aleph
### 2. Publish the file on aleph.im

```shell
aleph file pin ./data.squashfs
Expand All @@ -51,7 +51,7 @@ pip3 install -t ./packages -r ./requirements.txt
mksquashfs ./packages packages.squashfs
```

### 3. Publish the file on Aleph
### 3. Publish the file on aleph.im

```shell
aleph file pin ./packages.squashfs
Expand Down
4 changes: 2 additions & 2 deletions docs/overview.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Overview

## What is Aleph?
## What is aleph.im?

Aleph is an open-source off-chain P2P (peer-to-peer) network featuring a decentralized database (inc. file storage),
Aleph.im is an open-source off-chain P2P (peer-to-peer) network featuring a decentralized database (inc. file storage),
computing, and a DID framework. Aleph allows your blockchain-enabled app (dApp) to securely access trusted off-chain
data or computation through cross-chain connectivity and our client libraries.