Robot to build gentoo binpkg cleanly, continuously, automatically, with a support of flag variant.
Ruby Python Shell
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
lib
script
share/libexec
spec
.gitignore
.rspec
.travis.yml
Gemfile
LICENSE.txt
README.md
Rakefile
binpkgbot.gemspec

README.md

binpkgbot: Robot builds Gentoo binpkg in clean environment, automatically and continuously.

When you manage many Gentoo servers, I guess you're tired to run emerge -uDN @world and wait for all servers, or to install same packages in your all servers. Portage has binpkg feature, which allows to re-use prebuilt binaries when emergeing packages.

Binpkgbot allows you to build binpkg in clean environment, automatically and continuously! Continuous building of the latest packages also allows you to run -uDN anytime you want.

Features

  • Build in clean environment
    • Runs emerge on clean stage3 using systemd-nspawn every run
    • Fetch and install existing binpkg of build dependencies
    • Builds binpkg of new dependencies
  • 2 build modes:
    • Attempt to upgrade existing stage3 directory
    • Just install additional packages

Prerequisites

  • systemd based Linux box
  • systemd-nspawn
    • with btrfs filesystem

Installation

$ gem install binpkgbot

Usage

Set up

  1. Prepare stage3 (or 4) directory (e.g. /mnt/vol/stage)
  • binpkgbot adds modification when running upgrades. Otherwise it's used as base of ephemeral container
  • this directory should be in a btrfs filesystem
  1. /etc/portage directory outside of (1) (e.g. /mnt/vol/etc-portage)
  • binpkgbot always copies the entire directory into a container
  1. gentoo portage repository (e.g. /usr/portage)
  • binpkgbot syncs the specified directory; if this behavior is acceptable, you can specify /usr/portage.

then write some yml:

# binpkgbot.yml
stage: /mnt/vol/stage
etc_portage: /mnt/vol/etc-portage
portage_repo: /usr/portage

# use_sudo_for_nspawn: true

# emerge_options:
#   - '-v'
# config_protect_mask: false

## bind mounts for a container
# binds:
#   - /opt/my-overlay # default to read-only
#   - rw: /tmp/something_writable # or 'ro:'

#   - from: /mnt/vol/packages
#     to: /usr/portage/packages
#     writable: true
#   - from: /mnt/vol/elog
#     to: /var/log/portage/elog
#     writable: true

tasks:
  # Upgrade the stage (-uDN)
  - upgrade: '@world' # emerge -uDN @world
  - install:
      atom: '@preserve-rebuild'
      persist: true # Run sd-nspawn without --ephemeral option, default to false.


  # Simple way
  - install: 'sys-apps/dstat'
  # Complex way
  - install:
      atom: 'media-apps/ffmpeg'
      ## Using these flag modifier (:use, :accept_keywords, :unmasks, :masks) to define variants.
      use: x265
      ## or
      # use:
      #   - x265
      #   # to other packages
      #   - media-libs/x265 numa
      accept_keywords: true # default to ~*
      # accept_keywords:
      #   - true
      #   # - "~amd64"
      #   ## to other packages
      #   - "media-libs/x265 ~amd64"
      #   - media-libs/x265 # or default: ~*
      unmask: true # unmask a specified atom itself
      # unmask:
      #   - true
      #   - media-libs/x265
      # mask: 
      #   - media-libs/x264

  - include: ./task.yml # use file instead
  - include: ./task.d/* # or glob files and run all of them (order by filename)

## run something inside or outside a container
#   - run:
#       ## run outside of container? (default to false)
#       # host: true
#       ## persist change in a container (default to true)
#       # persist: false
#       script:
#         - emerge-webrsync
# ./task.yml
- install: ...

Running

$ binpkgbot
$ binpkgbot -c path/to/binpkgbot.yml

$ binpkgbot --help
$ binpkgbot --version

Recommended practices

  • Turn on binpkg-multi-instance and getbinpkg in $FEATURES
    • These are optional. Put them by yourself into make.conf.
  • Maintain different /etc/portage directory then put all common use flags, keywords, and unmaskings in the directory.
    • minimize USE flag difference. Use use: accept_keywords: unmask: for a case needs to make a variant of builds.

      (e.g. app-analyzer/zabbix has a flag to build server or go only with agent. Server needs both, but most servers are okay with having agent only. Then we need 2 variants in zabbix binpkg, so use use: configuration to build variants.)

      - install:
          atom: app-analyzer/zabbix
          use:
            - agent
            - server
      - install:
          atom: app-analyzer/zabbix
          use:
            - agent

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/sorah/binpkgbot.

License

The gem is available as open source under the terms of the MIT License.