Skip to content
Permalink
Browse files

Monorepo (#13)

Move the whole Coinboot project to a monorepo
  • Loading branch information
frzb committed Dec 3, 2019
1 parent 8aa3bc6 commit fab038685ed31d38ced65c81807bea52e98ffe0e
Showing with 4,550 additions and 236 deletions.
  1. +59 −66 .travis.yml
  2. +7 −1 README.md
  3. +4 −7 Vagrantfile
  4. +1 −0 boot
  5. +49 −0 coinbootmaker/README.md
  6. 0 {plugins → coinbootmaker/build}/.keep
  7. +423 −0 coinbootmaker/coinbootmaker
  8. +113 −0 coinbootmaker/src/claymores-miner
  9. +97 −0 coinbootmaker/src/ethminer
  10. +26 −0 coinbootmaker/src/node-exporter
  11. +16 −3 example_plugins/src/coinboot-vagrantbox → coinbootmaker/src/vagrantbox
  12. +1 −0 conf
  13. +43 −0 debirf/.travis.yml
  14. +674 −0 debirf/LICENSE
  15. +71 −0 debirf/README.md
  16. +51 −0 debirf/Vagrantfile
  17. 0 debirf/build/.keep
  18. +84 −0 debirf/build_and_run_images
  19. +68 −0 debirf/create_rootfs_and_kernel
  20. +952 −0 debirf/debirf
  21. +16 −0 debirf/docker-compose.yml
  22. +10 −0 debirf/notizen
  23. +69 −0 debirf/profiles/coinboot/debirf.conf
  24. +86 −0 debirf/profiles/coinboot/modules/a0_prep-root
  25. +96 −0 debirf/profiles/coinboot/modules/a1_install-kernel
  26. +64 −0 debirf/profiles/coinboot/modules/a2_r8168-network-driver
  27. +80 −0 debirf/profiles/coinboot/modules/a3_i915_bpo_driver
  28. +32 −0 debirf/profiles/coinboot/modules/coinboot-scripts
  29. +35 −0 debirf/profiles/coinboot/modules/create_user
  30. +27 −0 debirf/profiles/coinboot/modules/openssh-server
  31. +26 −0 debirf/profiles/coinboot/modules/systemd-serial-terminal
  32. +40 −0 debirf/profiles/coinboot/modules/x0_work_around_dhclient_hostname
  33. +30 −0 debirf/profiles/coinboot/modules/x1_motd
  34. +37 −0 debirf/profiles/coinboot/modules/y0_network
  35. +24 −0 debirf/profiles/coinboot/modules/z0_remove-locales
  36. +80 −0 debirf/profiles/coinboot/modules/z1_clean-root
  37. +592 −0 debirf/profiles/ubuntu-xenial-16.04-cloudimg-console.log
  38. BIN debirf/scripts/.create_plugin.py.swp
  39. +105 −0 debirf/scripts/create_plugin
  40. +107 −0 debirf/scripts/create_plugin.py
  41. +79 −0 debirf/scripts/dpkg_status.py
  42. +0 −65 docker-compose.yml
  43. +1 −0 docker-compose.yml
  44. BIN example_plugins/builds/coinboot-vagrantbox.tar.gz
  45. +0 −75 example_plugins/src/ethminer
  46. BIN img/preview_talk_asg19.png
  47. +1 −0 plugins
  48. +1 −1 { → server}/boot/coinboot.ipxe
  49. 0 { → server}/conf/dnsmasq/coinboot.conf
  50. +1 −1 { → server}/conf/environment/default.env
  51. 0 { → server}/conf/graylog/contentpacks/coinboot-remote-logging.json
  52. +7 −15 docker-compose_travis-ci.yml → server/docker-compose.yml
  53. 0 { → server}/docker/Dockerfile
  54. 0 { → server}/docker/coinboot-download-helper
  55. +0 −2 { → server}/docker/dnsmasq/dnsmasq.conf
  56. 0 { → server}/docker/graylog-contentpack-helper
  57. 0 { → server}/docker/nginx/default.conf
  58. 0 { → server}/docker/supervisor/coinboot.ini
  59. 0 server/plugins/.keep
  60. +165 −0 server/run_coinboot
@@ -1,67 +1,60 @@
dist: xenial
sudo: required
language: c
cache: ccache
---
env:
- KERNEL=4.15.0-43-generic
- KERNEL=4.15.0-45-generic
before_install:
# always quit the build something fails with a non-zero return value
#- set -e
- sudo apt update
- sudo apt install --yes sshpass build-essential pkg-config jq qemu-system-x86 ovmf iputils-ping
- CURL='curl --max-time 5 --retry-max-time 20 --retry 999'
- RESPONSE=$($CURL --silent "https://api.github.com/repos/frzb/coinboot-debirf/tags")
- sleep 5
- >
while ! RELEASE=$(echo $RESPONSE | jq -r '.[0].name'); do
echo "Calling the Github API has failed, repeat ..."
RESPONSE=$(curl --silent "https://api.github.com/repos/frzb/coinboot-debirf/tags")
sleep 5
done
- echo "Latest release is $RELEASE"
- echo $PATH
- sudo mkdir -p /etc/qemu
- echo 'allow all' | sudo tee /etc/qemu/bridge.conf
script:
- sudo docker-compose -f docker-compose_travis-ci.yml up -d
- sudo docker-compose exec coinboot env
- sudo docker-compose ps
#- sudo docker-compose logs -f coinboot
- while ! [ $(curl --silent http://192.168.1.2 | jq length) -gt 4 ]; do curl --silent http://192.168.1.2 | jq '.[].name'; echo '------------'; sleep 10; done
#- sudo docker-compose logs -f coinboot
# This line can be used for debugging Qemu iPXE purposes.
#- sudo qemu-system-x86_64 -m 2048 -smp 2 -nographic -serial mon:stdio -boot n -net nic,model=e1000 -net bridge,br=$(ip a|grep 192.168.1.1/24 | grep -oP br-.*)
# Boot with legacy BIOS
- >
sudo qemu-system-x86_64
-m 2048
-smp 2
-display none
-daemonize
-boot n
-net nic,model=e1000
-net bridge,br=$(ip a|grep 192.168.1.1/24 | grep -oP br-.*)
- while ! ping -c 1 192.168.1.10; do echo 'Waiting for Coinboot machine to respond to our ICMP echo requests ...'; sleep 10; done
- ping -c 10 192.168.1.10
- while ! nc -z 192.168.1.10 22; do echo 'Waiting for Coinboot machine to listen on port 22/SSH ...'; sleep 5; done
- sshpass -p ubuntu ssh -v -o StrictHostKeyChecking=no ubuntu@192.168.1.10 "grep -C 10 ${RELEASE} /etc/motd && uname -a"
- sudo killall qemu-system-x86_64
- while nc -z 192.168.1.10 22; do echo 'Waiting for Coinboot machine to stop listen on port 22/SSH ...'; sleep 5; done
# Boot with UEFI/OVMF
# This line can be used for debugging Qemu iPXE purposes.
# - sudo qemu-system-x86_64 -m 2048 -smp 2 -serial mon:stdio -curses -boot n -net nic,model=e1000 -net bridge,br=$(ip a|grep 192.168.1.1/24 | grep -oP br-.*) -bios /usr/share/OVMF/OVMF_CODE.fd
- >
sudo qemu-system-x86_64
-m 2048
-smp 2
-display none
-daemonize
-boot n
-net nic,model=e1000
-net bridge,br=$(ip a|grep 192.168.1.1/24 | grep -oP br-.*)
-bios /usr/share/OVMF/OVMF_CODE.fd
- while ! ping -c 1 192.168.1.10; do echo 'Waiting for Coinboot machine to respond to our ICMP echo requests ...'; sleep 10; done
- ping -c 10 192.168.1.10
- while ! nc -z 192.168.1.10 22; do echo 'Waiting for Coinboot machine to listen on port 22/SSH ...'; sleep 10; done
- sshpass -p ubuntu ssh -v -o StrictHostKeyChecking=no ubuntu@192.168.1.10 "grep -C 10 ${RELEASE} /etc/motd && ls -la /sys/firmware/efi && uname -a"
global:
# We use backticks in favor of the proper $() subshell call to cope with the
# evaluation of Travis CI. Else we had trouble with the braces of the regex.
# The regex matches on subdirectories only - identified by the trailing /.
- TO_BUILD=`git diff --name-only $TRAVIS_COMMIT_RANGE HEAD | grep -oP '^.*?(?=\/)' | uniq | tr -s "\n" ":" || exit 0`

jobs:
include:
- &build_and_run
stage: build and run images
# Only run this job when there are changes in the subdir 'debirf'
# Attention: Travis CI is only parsing this file if the condition is met.
# before_install: git diff --name-only $TRAVIS_COMMIT_RANGE HEAD | grep -qP '^debirf/' && env
before_install: env
# Verbose if conditional instead pure [[ ]] - else this won't work in this YAML file.
before_script: if ! [[ "$TO_BUILD" = *"debirf"* ]]; then exit 0; fi
env: KERNEL=4.15.0-45-generic SUITE=xenial
dist: xenial
sudo: required
language: bash
script: ./debirf/build_and_run_images
after_success:
- git config --global user.email "build@travis-ci.com"
- git config --global user.name "Travis CI"
- git tag -a "$(date +'%Y%m%d')" -m 'Automatic release by Travis CI'
- git push "https://${TOKEN}@github.com/${TRAVIS_REPO_SLUG}".git --tags
- git fetch origin
- git tag
deploy:
provider: releases
api_key: ${TOKEN}
file_glob: true
file: build/*
skip_cleanup: true
on:
branch: master
draft: true

- <<: *build_and_run
env: KERNEL=4.15.0-45-generic SUITE=xenial

- <<: *build_and_run
env: KERNEL=5.0.0-13-generic SUITE=disco

- &run_coinboot
stage: run coinboot
before_script: if ! [[ "$TO_BUILD" = *"server"* ]]; then exit 0; fi
env: KERNEL=4.15.0-45-generic SUITE=xenial
dist: xenial
sudo: required
language: bash
script: ./server/run_coinboot

#- <<: *run_coinboot
# env: KERNEL=4.15.0-45-generic SUITE=xenial

- <<: *run_coinboot
env: KERNEL=5.0.0-13-generic SUITE=disco
@@ -45,6 +45,12 @@ Its core features are:
This repository contains the Coinboot Server Docker container.
This container includes all services to get Coinboot up and running and boot diskless Coinboot Worker nodes over network.

### Talk at All Systems Go! 2019 conference

#### Coinboot - Cost effective, diskless GPU clusters for blockchain hashing and beyond

[![](img/preview_talk_asg19.png)](http://www.youtube.com/watch?v=6T9pBa9I_AQ "Coinboot - Cost effective, diskless GPU clusters for blockchain hashing and beyond")

## Requirements

Docker
@@ -84,7 +90,7 @@ There are also mandatory environment variables which are required to be configur

The RootFS and Kernel are downloaded autmatically when the Coinboot Server Docker container is started based on the `RELEASE`set at `./conf/environment/default.env`. Out of the box the latest available release is used.

You can **build** your own Coinboot base image using: [coinboot-debirf](https://github.com/frzb/coinboot-debirf).
You can **build** your own Coinboot base image using: [coinboot-debirf](https://github.com/frzb/coinboot/tree/master/debirf).
Or **download** a pre-build daily release at: https://github.com/frzb/coinboot-debirf/releases
These builds are made daily to contain all current packages updates and security fixes.
The RootFS (`*initramfs*`) and Kernel (`*vmlinuz*`) you want to use are to be placed in the directory `./boot`.
@@ -14,12 +14,6 @@ if ! which docker-compose; then
sudo chmod +x /usr/local/bin/docker-compose
fi
if ! docker-compose -f $COMPOSEFILE ps | grep -q coinboot; then
docker-compose -f $COMPOSEFILE up -d --build --force-recreate
else
docker-compose -f $COMPOSEFILE restart
fi
# Configure forwading and NAT cause the DHCP server vagrant box acts currently
# also as gateway.
# Ignore the masquerading set up for Docker (destination 172.17.0.0/16).
@@ -86,6 +80,8 @@ Vagrant.configure(2) do |config|
config.vm.define "coinboot-server" do |machine|
machine.vm.box = "bento/ubuntu-16.04"
machine.vm.provision "shell", inline: $coinboot_docker
machine.vm.provision "shell", inline: "/vagrant/server/run_coinboot"
machine.vm.network "forwarded_port", guest: 5900, host: 5900
interfaces = []

Socket.getifaddrs.each do |addr_info|
@@ -107,7 +103,8 @@ Vagrant.configure(2) do |config|
machine.vm.provision "shell", inline: 'rm -fv /vagrant/plugins/coinboot-vagrantbox.tar.gz'
else
machine.vm.network "private_network", ip: "192.168.1.2"
machine.vm.provision "shell", inline: 'cp -v /vagrant/example_plugins/builds/coinboot-vagrantbox.tar.gz /vagrant/plugins/coinboot-vagrantbox.tar.gz'
# FIXME: Path has changed in monorepo
# machine.vm.provision "shell", inline: 'cp -v /vagrant/example_plugins/builds/coinboot-vagrantbox.tar.gz /vagrant/plugins/coinboot-vagrantbox.tar.gz'
end
# Using '82540EM' provides 1GBit/s interface not just the default
# 100MBit/s one.
1 boot
@@ -0,0 +1,49 @@
# Coinbootmaker

`coinbootmaker` - a little helper to build your own Coinboot plugins.

## Requirements

* Docker

* a Debian or Ubuntu build host

* `brctl` which is part of the `bridge-utils` package

## Usage

```
coinbootmaker [-i] -p <file name> <path to initramfs>
-i Interactive mode - opens a shell in the build environment
-p <file name> Plugin to build
-h Display this help
```

### Example:

```
$ ./coinbootmaker /tmp/coinboot-initramfs-4.15.0-43-generic -p ethminer
```

`coinbootmaker` takes a path to a Coinboot-Initramfs to create an environment for building the plugins
by converting the given Initramfs into a Container image and run it.
A plugin creation script located at `./src` is executed in that `coinbootmaker` container and the resulting
plugin archive is written to the `./build` directory.

## License

MIT

Please notice even while the scripts to create Coinboot plugins are licensed with the MIT license the software packaged by this scripts may be licensed by an other license with different terms and conditions.
You have to agree to the license of the packaged software to use it.

## Author

Gunter Miegel
gm@coinboot.io

## Contribution

Fork this repo.
Make a pull request to this repo.
File renamed without changes.

0 comments on commit fab0386

Please sign in to comment.
You can’t perform that action at this time.