Skip to content

mirror

Jump to bottom
Jesús Daniel Colmenares Oviedo edited this page May 11, 2024 · 2 revisions

Creating a private AppJail mirror

AppJail has its own mirrors where images can be downloaded. They are publicly available, but in some situations it is preferable to create a mirror for private use. For security reasons, your company requires images to be generated by themselves, needs better control, or simply wants to get a mirror so as not to depend on AppJail mirrors.

Requirements

Don't hesitate to choose other alternatives that perform the same task as the following tools:

  • Gitea: A git hosting to put Makejails, the files it uses and the AJSPEC files.
  • Darkhttpd: A web server where images can be downloaded.
  • SSH: We need to remotely access the server, but SSH will mainly be used to upload images.

Except for the SSH server, which I assume is the same one that comes with FreeBSD, we can use appjail-director to install Gitea and Darkhttpd. Last but not least, we will use appjail-reproduce to create an image, which is much easier and efficient (in usability) than creating the image for each tag and writing the AJSPEC file ourselves.

Maybe you will find it useful to know my system information:

version:

# appjail version
3.3.0+fbe8792bc494bc94cedf112917db5889bb1c2cca
# appjail-director --version
appjail-director, version 0.8.0
# appjail-reproduce -v
0.4.0

releases available:

# appjail fetch list
ARCH   VERSION       NAME
amd64  14.0-RELEASE  default

/etc/rc.conf:

# CLEAR TEMP. DIRECTORY
clear_tmp_enable="YES"

# SYSLOGD
syslogd_flags="-ss"

# HOSTNAME
hostname="centralita.lan"

# KEYMAP
keymap="es.acc.kbd"

# Ethernet card configuration
ifconfig_re0_name="jext"
ifconfig_jext="DHCP"

# SSHD
sshd_enable="YES"

# NTPD
ntpd_sync_on_start="YES"

# MOUSED
moused_nondefault_enable="NO"   

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"

# ZFS
zfs_enable="YES"

# PF
pf_enable="YES"
pflog_enable="YES"

# IP Forwarding
gateway_enable="YES"

#  MOUSED
moused_nondefault_enable="NO"   

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"

# ZFS
zfs_enable="YES"

# PF
pf_enable="YES"
pflog_enable="YES"

# IP Forwarding
gateway_enable="YES"

# DNS
dnsmasq_enable="YES"
dnsmasq_conf="/usr/local/share/appjail/files/dnsmasq.conf"

# CLONED INTERFACES
cloned_interfaces="tap0"

# AppJail (DNS)
appjail_dns_enable="YES"
ifconfig_tap0_name="ajdns"
ifconfig_ajdns="inet 172.0.0.1/32"

# AppJail
appjail_enable="YES"

# AppJail (healthcheckers)
appjail_health_enable="YES"

# AppJail (NAT per network)
appjail_natnet_enable="YES"

/usr/local/etc/appjail/appjail.conf:

EXT_IF=jext
ON_IF=jext
FREEBSD_VERSION=14.0-RELEASE
FREEBSD_ARCH=amd64
IMAGE_ARCH=amd64
SHORTEN_DOMAIN_NAMES=1
ENABLE_ZFS=1
ENABLE_DEBUG=0
TAR_XZ_ARGS="--xz --options xz:threads=0"
TAR_ZSTD_ARGS="--zstd --options zstd:threads=0"
DEFAULT_RESOLV_CONF="/usr/local/etc/appjail/resolv.conf"

/usr/local/etc/appjail/resolv.conf:

nameserver 172.0.0.1

.config/appjail-reproduce/config.conf:

MIRRORS="http://192.168.1.107"
COMPRESS_ALGO=zstd

/usr/local/etc/doas.conf:

permit nopass keepenv :appjail as root cmd appjail
permit nopass :appjail as root cmd appjail-config

/etc/pf.conf:

nat-anchor "appjail-nat/jail/*"
nat-anchor "appjail-nat/network/*"
rdr-anchor "appjail-rdr/*"

/boot/loader.conf:

kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
cryptodev_load="YES"

# ZFS.
zfs_load="YES"

# Enable RACCT.
kern.racct.enable=1

appjail-director.yml:

options:
  - virtualnet: ':<random> default'
  - nat:

services:
  git-hosting:
    name: gitea
    makejail: gh+AppJail-makejails/gitea
    environment:
      - GITEA__SERVER__DOMAIN: '192.168.1.107'
      - GITEA__DEFAULT__APP_NAME: 'Welcome to my AppJail mirror!'
    options:
      - expose: '3000'
    arguments:
      - gitea_tag: '14.0'
    volumes:
      - gitea-db: gitea-db
      - gitea-git: gitea-git
  web-server:
    name: darkhttpd
    makejail: gh+AppJail-makejails/darkhttpd
    options:
      - expose: '80'
    arguments:
      - darkhttpd_tag: '14.0'
    volumes:
      - wwwdir: '/usr/local/www/darkhttpd'

default_volume_type: '<volumefs>'

volumes:
  wwwdir:
    device: .volumes/wwwdir
    type: nullfs
  gitea-db:
    device: .volumes/gitea-db
  gitea-git:
    device: .volumes/gitea-git

.env:

DIRECTOR_PROJECT=appjail-mirror

However, before running all the commands described in this document, you should familiarize yourself with AppJail and its terms, especially the following:

  • Images: Handbook, appjail-image(1).
  • AJSPEC: Handbook, appjail-ajspec(5).
  • Makejails: Handbook, appjail-makejail(1), appjail-makejail(5).

Preparing the Project

Let's create our Director project:

# appjail-director up
Starting Director (project:appjail-mirror) ...
Creating web-server (darkhttpd) ... Done.
Creating git-hosting (gitea) ... Done.
Finished: appjail-mirror
# appjail-director info
appjail-mirror:
 state: DONE
 last log: /root/.director/logs/2024-05-10_17h44m07s
 locked: false
 services:
  + web-server (darkhttpd)
  + git-hosting (gitea)

Mirroring an image

  1. Enter the address in your browser where Gitea listens. Since I am using a different computer to host the Director project than the one I use to write this article, I put the external address http://192.168.1.107:3000.

  1. Create a new account on your Gitea instance.

  1. Wait until the home page loads completely.

  1. Let's create a new migration in "+ > New Migration."

  1. We will use a repository hosted on Github, so select it.

  1. You can use any repository you want, but for simplicity I will use hello, which is designed for educational purposes.

  1. Press "Migrate Repository"

  1. Wait until the repository home page loads completely.

  1. Copy the repository URL.

  1. Clone the repository using git-clone(1), cd into the repository directory, identify the files that need to be changed and see where the current images are.

  1. Download the image to the mirror server, specifically where the web server listens.

  1. Change the files to point to the new URL.

  1. Commit and push the changes.

  1. Test.

Making changes to an image

Our examples only consider images created by a third-party, but what happens when we need to build an image ourselves? A common example is when we need to rebuild an image periodically (e.g. a week, a month, etc.) to reinstall the project with its updated dependencies.

  1. Clone Reproduce's projects.
# git clone git@github.com:DtxdF/reproduce-projects.git .reproduce/projects
  1. Build the project.
# appjail-reproduce -b hello
[ info  ] Started at Fri May 10 21:24:44 -04 2024
[ info  ] [hello]:
[ info  ] Logs: /home/dtxdf/.reproduce/logs/hello/2024-05-10_21h24m44s.log
[ info  ] > [hello] (osarch:amd64, osversion:13.3-RELEASE, tag:13.3):
[ info  ] Executing Makejail: jail:reproduce_9a220842-baa5-46e0-a531-5a84325d1211, release:default, args:
[ info  ] Execution time: 00:00:07
[ info  ] Removing: rm -f var/log/*
[ info  ] Removing: rm -f var/cache/pkg/*
[ info  ] Removing: rm -rf usr/local/etc/pkg
[ info  ] Removing: rm -f var/run/* 2> /dev/null || :
[ info  ] Exporting hello
[ info  ] Export time: 00:00:01
[ info  ] > [hello] (osarch:amd64, osversion:14.0-RELEASE, tag:14.0):
[ info  ] Executing Makejail: jail:reproduce_9a220842-baa5-46e0-a531-5a84325d1211, release:default, args:
[ info  ] Execution time: 00:00:06
[ info  ] Removing: rm -f var/log/*
[ info  ] Removing: rm -f var/cache/pkg/*
[ info  ] Removing: rm -rf usr/local/etc/pkg
[ info  ] Removing: rm -f var/run/* 2> /dev/null || :
[ info  ] Exporting hello
[ info  ] Export time: 00:00:01
[ info  ] ---
[ info  ] Ended at Fri May 10 21:25:08 -04 2024
[ info  ] Build time: 00:00:24
[ info  ] Hits: 2
[ info  ] Errors: 0
  1. Copy the new AJSPEC.
# cd /tmp/hello
# cp -f /usr/local/appjail/cache/images/hello/.ajspec .ajspec
  1. Commit and push the changes.
# git commit -am 'Update AJSPEC'
[main 03df5c3] Update AJSPEC
 1 file changed, 8 insertions(+), 8 deletions(-)
# git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 556 bytes | 556.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
remote: . Processing 1 references
remote: Processed 1 references in total
To http://192.168.1.107:3000/DtxdF/hello.git
   d410929..03df5c3  main -> main
  1. Update the image.
# ssh control-centralita appjail fstab jail darkhttpd list -n 0 device
DEVICE
/root/jails/gitea/.volumes/wwwdir
# scp /usr/local/appjail/cache/images/hello/*.appjail root@192.168.1.107:/root/jails/gitea/.volumes/wwwdir/hello
13.3-amd64-image.appjail                                                                                                                100%  866KB  10.6MB/s   00:00    
14.0-amd64-image.appjail                                                                                                                100%  871KB  10.7MB/s   00:00

Afterword

As you can see, creating a mirror is very simple, but that's simply because AppJail automates the hard parts.

We use the hello project for educational purposes, a simplistic approach but one that works well as you can apply the same principles to other Makejails, not only existing ones, but also new ones.

Clone this wiki locally