Skip to content
Script for creating custom OS images for single-board computers
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.

README.md

build

Pieman Tweet

Pieman

Pieman is a script for creating custom OS images for single-board computers such as Raspberry Pi and Orange Pi. The authors of Pieman were inspired by the project named rpi23-gen-image. The main reason why it was decided to create a new project instead of improving the existing one is that rpi23-gen-image is intended for creating images only for Raspberry Pi 2 and 3 based on Debian GNU/Linux. Unfortunately, it doesn't have a way to be extended to support different operating systems and single-board computers. Improving the situation would require significant efforts to rework the codebase of the project.

The project is named after a superhero Pie Man appeared in the Simpsons' episode Simple Simpson.

Pieman is a core component of CusDeb.

Features

Table of Contents

Supported Devices and OSes

Alpine 3.9 Debian 10 «Buster» Devuan 1 «Jessie» Kali Linux Rolling Raspbian 9 «Stretch» Ubuntu 16.04 «Xenial Xerus» Ubuntu 18.04 «Bionic Beaver»
Orange Pi PC Plus 32bit 32bit
Orange Pi Zero 32bit 32bit
Raspberry Pi Model B and B+ 32bit
Raspberry Pi 2 Model B 32bit 32bit 32bit 32bit 32bit
Raspberry Pi 3 Model B 32bit 32bit 32bit 32bit, 64bit
Raspberry Pi Zero 32bit

Getting Started

Dependencies

Mandatary

  • Development tools
    • GNU Bison
    • C programming language compiler
    • Flex
    • Git
    • Make
    • SWIG
  • Python
    • Development libraries and header files related to Python 2.7
    • Python 2.7
    • Python 3 (3.5 or higher)
    • Setuptools
  • Utils
    • dosfstools
    • dpkg
    • GNU Parted
    • GnuPG
    • mkpasswd
    • pandoc
    • rsync
    • uuidgen
    • wget
    • which
    • xz

Here are the commands to install the mandatory dependencies

  • on Debian or Ubuntu
    $ sudo apt-get install bison dosfstools flex gcc git gnupg make pandoc parted python-dev python3-pip python3-setuptools swig rsync uuid-runtime wget whois xz-utils
    
  • on Fedora
    $ sudo dnf install bison dosfstools dpkg expect flex gcc git gpg make pandoc parted python2-devel python3-pip python3-setuptools rsync swig wget which xz
    

Optional

  • To enable COMPRESS_WITH_BZIP2: bzip2
  • To enable Mender support:
    • Development libraries and header files related to C standard library (make sure the package, which is going to be installed to satisfy the dependency, includes /usr/include/sys/types.h)
    • Go programming language compiler
    • bc
    • dtc

Here are the commands to install the optional dependencies

  • on Debian or Ubuntu
    sudo apt-get install bzip2 bc gcc device-tree-compiler golang libc6-dev-i386
    
  • on Fedora
    sudo dnf install bc bzip2 dtc golang
    

Supported platforms

Theoretically, Pieman can be run on any GNU/Linux, however, it was very carefully tested only on:

  • Debian 9 «Stretch»
  • Fedora 29
  • Ubuntu 16.04 «Xenial Xerus»
  • macOS 10.13 «High Sierra» (running Pieman in the official Docker container)

Installation

Docker

First, get the latest Pieman Docker image from Docker Hub.

$ docker pull cusdeb/pieman

Then, get docker-pieman.sh

$ wget https://raw.githubusercontent.com/tolstoyevsky/pieman/master/docker-pieman.sh -O docker-pieman.sh

or using curl

$ curl -O https://raw.githubusercontent.com/tolstoyevsky/pieman/master/docker-pieman.sh

Note, that the script requires bash 4 or higher, so macOS users should upgrade their bash if they haven't done it yet.

Usage

Simply run docker-pieman.sh to create an image based on Raspbian Stretch for Raspberry Pi 3.

$ chmod +x docker-pieman.sh
$ ./docker-pieman.sh

Under the hood the script runs

$ docker run --privileged --rm -v $(pwd):/result -v /dev:/dev cusdeb/pieman

It's quite wordy, isn't it? docker-pieman.sh is intended to hide the details and provide an easy-to-use command-line interface.

Another example shows how to create an image based on Ubuntu Xenial for Raspberry Pi 2 with htop and mc pre-installed

$ ./docker-pieman.sh -e DEVICE=rpi-2-b -e OS=ubuntu-xenial-armhf -e INCLUDES=htop,mc

The built images will be located in the current directory. By the way, you can specify the name of your project via the PROJECT_NAME environment variable. You can find details on DEVICE, INCLUDES, OS, PROJECT_NAME and other environment variables (called parameters) which help customizing images in the Documentation section.

Manual

First, clone the Pieman git repo:

$ git clone https://github.com/tolstoyevsky/pieman.git

Then, install the Pieman mandatory dependencies.

Finally, install the required utilities and modules written in Python.

$ sudo pip3 install pieman
Usage

Go to the project directory and execute the following command to create an image based on Raspbian Stretch for Raspberry Pi 3:

$ sudo ./pieman.sh

To create an image based on Ubuntu Xenial for Raspberry Pi 2 with htop and mc pre-installed run

$ sudo env DEVICE=rpi-2-b OS=ubuntu-xenial-armhf INCLUDES=htop,mc ./pieman.sh

The built image will be located in the build directory. By the way, you can specify the name of your project via the PROJECT_NAME environment variable. You can find details on DEVICE, INCLUDES, OS, PROJECT_NAME and other environment variables (called parameters) which help customizing images in the Documentation section.

Running tests

The Pieman tests are divided into two parts: the tests for the package (which contains the utilities written in Python) and the tests for the Pieman script (written in Bash).

To run the tests for the package, create a virtual environment, activate it and install the Pieman requirements to it (suppose you are in the directory which contains the Pieman source code directory).

$ virtualenv -p python3 pieman-env
$ source ./pieman-env/bin/activate
$ pip install -r pieman/pieman/requirements.txt

Then run the tests from the Pieman source code directory in the following way:

$ env PYTHONPATH=$(pwd)/pieman python ./pieman/test/runtest.py

Editing the PYTHONPATH environment variable is necessary to run the tests, using the latest version of the package.

To run the tests for the Pieman script, you will need both the above-mentioned virtual environment (because the Pieman script tests depends on the package tests) and shUnit2. Install shUnit2 executing sudo apt-get install shunit2 on Debian/Ubuntu or sudo dnf install shunit2 on Fedora. Then, go to the test directory and run the tests in the following way:

$ env PIEMAN_UTILS_DIR=$(pwd)/../pieman/bin ./test_essentials.sh
$ env PIEMAN_UTILS_DIR=$(pwd)/../pieman/bin ./test_functions.sh

Editing the PIEMAN_UTILS_DIR environment variable is necessary to run the tests, using the latest version of the Pieman utilities.

Documentation

The operating system of the target image is specified via the OS environment variable. The next table maps full names of the supported operating systems to their short name intended for using as values of OS.

Full name Short name
Alpine 3.9 (32-bit) alpine-3.9-armhf
Debian 10 «Buster» (32-bit) debian-buster-armhf
Devuan 1 «Jessie» (32-bit) devuan-jessie-armhf
Kali Linux Rolling (32-bit) kali-rolling-armhf
Raspbian 9 «Stretch» (32-bit) raspbian-stretch-armhf
Ubuntu 16.04 «Xenial Xerus» (32-bit) ubuntu-xenial-armhf
Ubuntu 18.04 «Bionic Beaver» (32-bit) ubuntu-bionic-armhf
Ubuntu 18.04 «Bionic Beaver» (64-bit) ubuntu-bionic-arm64

The device the target image is created for is specified via the DEVICE environment variable. The next table maps full names of the supported devices to their short name intended for using as values of DEVICE.

Full name Short name
Orange Pi PC Plus opi-pc-plus
Orange Pi Zero opi-zero
Raspberry Pi Model B and B+ rpi-b
Raspberry Pi 2 Model B rpi-2-b
Raspberry Pi 3 Model B rpi-3-b
Raspberry Pi Zero rpi-zero

Parameters

General

OS="raspbian-stretch-armhf"

Specifies the operating system to be used as a base for the target image. You can find all the possible values for the parameter in the table above (see the "Short name" column).

DEVICE="rpi-3-b"

Specifies the device the image is going to be run on. You can find all the possible values for the parameter in the table above (see the "Short name" column).

BASE_DIR=""

Specifies the chroot environment to be used instead of creating a new one.

Note, that the parameter conflicts with CREATE_ONLY_CHROOT.

PROJECT_NAME="xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx"

Each image is built in in the context of some project. The parameter allows specifying the project name. By default, the name is a randomly generated UUID 32-character string.

BUILD_DIR="build"

Specifies the projects location. By default, the directory named build is created in the current directory during the build process with ownership specified via IMAGE_OWNERSHIP.

CREATE_ONLY_CHROOT=false

Restricts Pieman to only creating a chroot environment based on the operating system specified via OS. The chroot environment is stored in build/${PROJECT_NAME}/chroot and can be used immediately or later to reduce the time of building images. See BASE_DIR.

LOCALE="en_US.UTF-8"

Specifies the locale.

PIEMAN_DIR="$(pwd)"

Specifies the directory into which Pieman is installed.

PREPARE_ONLY_TOOLSET=false

Restricts Pieman to only preparing or upgrading the toolset which is located in the directory specified via TOOLSET_DIR.

TIME_ZONE="Etc/UTC"

Specifies the time zone of the system.

TOOLSET_CODENAME="v2-hermes"

Specifies the toolset codename. The parameter allows users and developers to switch between different toolsets. Each codename is connected to its directory in ${TOOLSET_DIR} which, in turn, contains the target toolset. When a codename is passed via ${TOOLSET_CODENAME} but there is no such directory in ${TOOLSET_DIR}, the process of creating of the directory and installing the toolset into it will be initiated.

Note that the default codename belongs to the latest toolset which supports all the Pieman features, so if you don't have any particular reason why you should change the codename, it's recommended to use the default one. However, if you decide to change the default codename, you should know how to later distinguish the official toolsets from your own ones. The official toolsets have the following naming convention: v{n}-{name} where {n} is an order number and {name} is a character name from the Futurama/Simpsons universe.

TOOLSET_DIR="${PIEMAN_DIR}/toolset"

Specifies the directory which contains the tools necessary for creating chroot environments based on Alpine Linux and different Debian-based distributions. The toolset consists of debootstrap and apk.static.

XFCE4="false"

Specifies whether to install xfce4 with lxdm. Available only on Ubuntu 18.04 Bionic Beaver 32-bit or 64-bit.


Networking

ENABLE_GOOGLE_DNS=false

Enables the most common DNS server provided by Google.

The DNS IP addresses (IPv4) are the following:

  • 8.8.8.8
  • 8.8.4.4

Note, that the parameter conflicts with ENABLE_BASIC_YANDEX_DNS, ENABLE_FAMILY_YANDEX_DNS and ENABLE_CUSTOM_DNS.

ENABLE_BASIC_YANDEX_DNS=false

Enables a quick and reliable DNS server provided by Yandex.

The DNS IP addresses (IPv4) are the following:

  • 77.88.8.8
  • 77.88.8.1

Note, that the parameter conflicts with ENABLE_GOOGLE_DNS, ENABLE_FAMILY_YANDEX_DNS and ENABLE_CUSTOM_DNS.

ENABLE_FAMILY_YANDEX_DNS=false

Enables the DNS server provided by Yandex with protection from "adult" content.

  • 77.88.8.7
  • 77.88.8.3

Note, that the parameter conflicts with ENABLE_GOOGLE_DNS, ENABLE_BASIC_YANDEX_DNS, and ENABLE_CUSTOM_DNS.

ENABLE_CUSTOM_DNS=""

Enables a custom DNS server specified via this parameter.

Note, that the parameter conflicts with ENABLE_GOOGLE_DNS, ENABLE_BASIC_YANDEX_DNS, and ENABLE_FAMILY_YANDEX_DNS.

HOST_NAME="pieman-${DEVICE}"

Specifies the hostname of a device.


Package manager

ALLOW_UNAUTHENTICATED=false

Specifies whether to install packages without checking their signatures.

ENABLE_COMMUNITY=false

This is an Alpine-specific parameter. It enables the community repository.

ENABLE_NONFREE=false

This is a Debian-specific parameter. It enables the non-free and contrib sections in /etc/apt/sources.list. By default, only the main section is used. Sections are also called components or areas.

ENABLE_UNATTENDED_INSTALLATION=false

Specifies whether to install packages without prompting the user to answer any questions.

ENABLE_UNIVERSE=false

This is an Ubuntu-specific parameter. It enables the universe section in /etc/apt/sources.list. By default, only the main section is used.

INCLUDES=""

A comma-separated list of the packages to be installed on the system specified via OS.


Users

ENABLE_SUDO=true

Specifies whether to install sudo. If ENABLE_USER is set to true, Pieman adds the user USER_NAME to /etc/sudoers.

ENABLE_USER=true

Specifies whether to create a non-root user USER_NAME with the password USER_PASSWORD.

PASSWORD="secret"

Specifies the root password. It's HIGHLY RECOMMENDED to change the default root password.

If PASSWORD equals to -, Pieman will prompt for a password, and read it without echoing to screen.

SUDO_REQUIRE_PASSWORD=true

Tells sudo whether it should prompt for the password.

It's necessary to disable the password prompts if you want to manage your device via, for example, SSH Button.

USER_NAME="cusdeb"

Specifies a non-root user name. It's ignored if ENABLE_USER is set to false.

USER_PASSWORD="secret"

Specifies a non-root user password. It's ignored if ENABLE_USER is set to false. It's HIGHLY RECOMMENDED to change the default user password.

If USER_PASSWORD equals to -, Pieman will prompt for a password, and read it without echoing to screen.


Image

COMPRESS_WITH_BZIP2=false

Compresses the resulting image using bzip2.

Note, that the parameter conflicts with COMPRESS_WITH_GZIP and COMPRESS_WITH_XZ.

COMPRESS_WITH_GZIP=false

Compresses the resulting image using gzip.

Note, that the parameter conflicts with COMPRESS_WITH_BZIP2 and COMPRESS_WITH_XZ.

COMPRESS_WITH_XZ=false

Compresses the resulting image using xz.

Note, that the parameter conflicts with COMPRESS_WITH_BZIP2 and COMPRESS_WITH_GZIP.

IMAGE_OWNERSHIP="$(id -u "$(stat -c "%U" "$0")"):$(id -g "$(stat -c "%G" "$0")")"

Specifies the ownership of the target image (see PROJECT_NAME) and project directory (see BUILD_DIR). By default, the ownership is borrowed from pieman.sh which, as a rule, belongs to a regular user.

Note, that the parameter must follow the format "uid:gid" where uid and gid are numbers.

IMAGE_ROOTFS_SIZE=0

Specifies the rootfs partition size in megabytes. Beware! Build will fail if rootfs doesn't fit into that size. If the parameter is equal to 0, the rootfs size will be calculated automatically.


Pieman-specific

PYTHON="$(which python3)"

Specifies the Python 3 interpreter to be used when calling the Pieman-specific utilities.

Extra

CREATE_ONLY_MENDER_ARTIFACT=false

Restricts Pieman to only creating an artifact (a file with the .mender extension) which can later be uploaded to hosted.mender.io to provide for OTA updates.

Note, that the parameter conflicts with CREATE_ONLY_CHROOT and ENABLE_MENDER.

ENABLE_BSC_CHANNEL=false

Specifies whether to run the build status codes server (also known as bscd). If the parameter is set to true, Pieman will check the connection to the Redis server (specified via REDIS_HOST and REDIS_PORT) and, in case there is no problem with that, it will be pushing the build status codes to the channel named bscd-${PROJECT_NAME}.

See the documentation page devoted to the build status codes server and client for the details.

ENABLE_MENDER=false

Specifies whether to install the Mender client to provide for OTA updates.

Note that the OTA updates support is currently limited to 32-bit Raspbian for Raspberry Pi 3 Model B.

MENDER_ARTIFACT_NAME="release-1_1.7.0"

The name of an image or update (called artifact) that will be built if either ENABLE_MENDER or CREATE_ONLY_MENDER_ARTIFACT is specified. Note that different updates must have different names.

MENDER_DATA_SIZE=128

Specifies the size in megabytes of the data partition.

The parameter is used only when ENABLE_MENDER is set to true.

MENDER_INVENTORY_POLL_INTERVAL=86400

Specifies the frequency (in seconds) for periodically sending inventory data. Inventory data is always sent after each boot of the device, and after a new update has been correctly applied and committed by the device in addition to this periodic interval. Default value: 86400 seconds (one day).

MENDER_RETRY_POLL_INTERVAL=300

Specifies the number of seconds to wait between each attempt to download an update file. Note that the client may attempt more often initially to enable rapid upgrades, but will gradually fall back to this value if the server is busy. See MENDER_INVENTORY_POLL_INTERVAL.

MENDER_TENANT_TOKEN=""

Specifies a token which identifies which tenant a device belongs to. It requires an account on hosted.mender.io.

The parameter is used only when ENABLE_MENDER is set to true.

MENDER_SERVER_URL="https://hosted.mender.io"

Specifies the server for the client to connect to.

MENDER_UPDATE_POLL_INTERVAL=1800

Specifies the frequency (in seconds) the client will send an update check request to the server. Default value: 1800 seconds (30 minutes).

REDIS_HOST="127.0.0.1"

Specifies the Redis server host used by the build status codes server (see ENABLE_BSC_CHANNEL).

REDIS_PORT=6379

Specifies the Redis server port used by the build status codes server (see ENABLE_BSC_CHANNEL).

Storing parameters in .env file

You can also specify the parameters in the .env file. In this case they will have the higher priority than the parameters provided via the command line.

Running images in emulator

It's possible to run the images built by Pieman in QEMU. The nearby project MMB simplifies the process. The project is the set of Dockerfiles and assets for building Docker images with different services. Now QEMU, which is one of the services, helps running the images based on Ubuntu 18.04 «Bionic Beaver» (64-bit) for Raspberry Pi 3.

Daily image builds

You can find the images of all supported operating systems for all supported devices here. Login username is cusdeb, password is secret.

FAQ

DNS resolving is broken in Kali Linux

According to the Kali Linux Policies, network services are disabled by default. To waken the DNS resolving from its sleep, run the following commands

sudo systemctl enable resolvconf
sudo systemctl start resolvconf

Authors

See AUTHORS.

Licensing

Pieman is available under the GNU General Public License version 3.

Pieman borrows some pieces of code from rpi23-gen-image which are available under the GNU General Public License version 2.

Contribute

If you want to contribute to Pieman, we encourage you to do so by sending a pull request.

In case you encounter any issues, please feel free to raise a ticket!

If you like Pieman and want to support it, then please star the project or watch it on GitHub.

paypal

You can’t perform that action at this time.