build_falter: add custom imagebuilder

This commit introduces the possibility to use a custom imagebuilder. It fixes #65. Signed-off-by: Martin Hübner <martin.hubner@web.de>
freifunk-berlin · Aug 5, 2021 · 399f99b · 399f99b
1 parent f980f82
commit 399f99b
Show file tree

Hide file tree

Showing 2 changed files with 79 additions and 32 deletions.
diff --git a/README.md b/README.md
@@ -5,15 +5,15 @@ The Falter project is split into different repositories. For the beginning there
 + **[repo_builder](https://github.com/Freifunk-Spalter/repo_builder)**: In that repo there is a script which compiles the source codes from *packages* repo into a package feed. We use the dockerized OpenWrt-SDK for that.
 + **[builter](https://github.com/Freifunk-Spalter/builter)**: The builter assembles Freifunk images from the OpenWrt-imagebuilder and the pre-compiled package feed from repo-builder. If you want to include a new app into falter, you'd need to add it to the packagelists defined here.
 
-
 # falter-builter
+
 This script packages falter-firmware from openwrt-imagebuilder and falter-feed. In comparison to the old buildsystem, it is almost insanely fast.
 
 ## Utilisation
 
 The script takes five positional arguments.
 
-```
+```sh
 ./build_falter [-p packageset] [-v version] [-t target] [-s subtarget] [-r router]
 ```
 
@@ -27,43 +27,60 @@ If you like to build only one specific router-profile, you *must* give all the a
 
 After the buildprocess finished, you will find the images in `firmwares/`.
 
-
 ## Quickstart: build your own image
 
 Lets assume you'd like to build a stable-release-tunneldigger-image for your GL-AR150 router. To achieve that, you should invoke the buildscript in that way:
 
-```
+```sh
 ./build_falter -p packageset/19.07/tunneldigger.txt -v 1.1.1 ath79 -s generic -r glinet_gl-ar150
 ```
+
 If you are more comfortable in memorizing it that way, you can use long arguments:
-```
+
+```sh
 ./build_falter --packageset packageset/19.07/tunneldigger.txt --version 1.1.1 --target ath79 --sub-target generic -router glinet_gl-ar150
 ```
 
 ### Find your routers profile
 
 If you don't know the profile name for your router, you should use `-l` to find it. That option will show you a list of all routers with there profiles. Pick the profile-name there and give it to the script with the `-r` parameter.
 
-```
+```sh
 ./build_falter -p packageset/19.07/tunneldigger.txt -v 1.1.1-snapshot -t ath79 -s generic -l
 ```
+
 In the router list you will find something similar like that:
-```
+
+```sh
 glinet_gl-ar150:
     GL.iNet GL-AR150
     SupportedDevices: glinet,gl-ar150 gl-ar150
 ```
+
 The profile name is at the first line. Omit the colon:
-```
+
+```sh
 ./build_falter -p packageset/19.07/tunneldigger.txt -v 1.1.1-snapshot -t ath79 -s generic -r glinet_gl-ar150
 ```
 
+### Use custom imagebuilder
+
+You can use your custom imagebuilder with falter to. This comes in handy for development and for using features, that are not in regular OpenWrt yet. To get that you should call builter like that:
+
+```sh
+./build_falter -v 1.2.0-snapshot -p packageset/21.02/tunneldigger.txt -i openwrt-imagebuilder-21.02.0-rc3-octeon.Linux-x86_64.tar.xz -r ubnt_edgerouter
+```
+
+NOTE: The imagebuilder must be in the root of builter-directory. It ist not supported to have it anywhere else.
+
 ## Use builter with buildbot
 
 For the image generation with buildbot, builter should be invoked like this:
-```
+
+```sh
 ./build_falter -p all -v <release> -t <target>
 ```
+
 The argument `all` will signalise builter to generate all three flavours of images. In `firmwares/` the images get sorted by package-list. Below the packagelist-directorys, the regular hirachy $TARGET/$SUBTARGET applies.
 
 CAUTION: Argument `all` will only work if at least release *and* target are specified.
diff --git a/build_falter b/build_falter
@@ -52,6 +52,9 @@ Options:
   -d|--development
     use development-feeds instead of release-feeds
 
+  -i|--image-builder
+    use custom image-builder instead of standart-openwrt ones.
+
 " >&2
 }
 
@@ -150,6 +153,18 @@ while :; do
         fi
         ;;
 
+    i*|-image-builder)
+        if [ "$2" ]; then
+            echo "Use Custom Imagebuilder at: $2"
+            IMAGE_BUILDER_PATH="$2"
+            shift
+        else
+            echo "Custom Imagebuilder parameters incomplete!" >&2
+            print_usage
+            exit 1
+        fi
+        ;;
+
     l* | -list-routers)
         PARSER_LIST_ROUTERS="y"
         ;;
@@ -187,6 +202,13 @@ while :; do
     fi
 done
 
+
+# ToDo: check for complete input:
+# -packagelists given?
+# falter-version given
+
+
+
 # check for dependencies.
 SCRIPT_DEPENDS="awk curl gawk grep git gettext python3 rsync sed unzip wget"
 for DEP in $SCRIPT_DEPENDS; do
@@ -299,24 +321,36 @@ function generate_embedded_files {
 }
 
 function start_build {
-    IMAGE_BUILDER_URL=$1
+    # use local imagebuilder if it was given
+    echo "$IMAGE_BUILDER_PATH"
+    echo "$1"
+    if [ -n "$IMAGE_BUILDER_PATH" ]; then
+      IMAGE_BUILDER_URL=$IMAGE_BUILDER_PATH
+    else
+      IMAGE_BUILDER_URL="$1"
+    fi
     local TMP=$2 # slice packageset-name from path
     local PKG_SET=$(echo $TMP | rev | cut -d'/' -f1 | rev | cut -d'.' -f1)
     local DEVICE=$3
     FILENAME=$(basename $IMAGE_BUILDER_URL)
     FOLDERNAME=$(basename $FILENAME .tar.xz)
     BRANCH=$(derive_branch_from_url $IMAGE_BUILDER_URL)
+    [ -z $BRANCH ] && BRANCH="snapshot"
     echo "building using: $IMAGE_BUILDER_URL"
     echo "selected branch: $BRANCH"
 
-    # store imagebuilders in cache. Reload, if there is a newer version avaiable
-    local CACHE="../imagebuilder_cache"
-    if [ ! -d $CACHE ]; then mkdir -p $CACHE; fi
-    cd $CACHE
-    wget -N --no-if-modified-since $IMAGE_BUILDER_URL
-    cd ../build
-    # pull imagebuilder from cache-dir
-    cp ../imagebuilder_cache/$FILENAME $FILENAME
+    if [ -z $IMAGE_BUILDER_PATH ]; then
+      # store imagebuilders in cache. Reload, if there is a newer version avaiable
+      local CACHE="../imagebuilder_cache"
+      if [ ! -d $CACHE ]; then mkdir -p $CACHE; fi
+      cd $CACHE
+      wget -N --no-if-modified-since $IMAGE_BUILDER_URL
+      cd ../build
+      # pull imagebuilder from cache-dir
+      cp ../imagebuilder_cache/$FILENAME $FILENAME
+    else
+      cp ../$IMAGE_BUILDER_PATH $FILENAME
+    fi
 
     echo $PWD
     echo "Extracting imagebuilder..."
@@ -333,6 +367,7 @@ function start_build {
         exit 0
     fi
 
+    # Target is in defferent position in the URL, depending on the OpenWrt version.
     case $BRANCH in
     snapshot)
         ispos=7
@@ -341,6 +376,10 @@ function start_build {
         ispos=8
         ;;
     esac
+    # when using custom imagebuilder, the target is on 8th position everytime.
+    if [ -n "$IMAGE_BUILDER_PATH" ]; then
+        ispos=8
+    fi
 
     INSTR_SET=$(grep "openwrt_base" repositories.conf | awk -F'/' "{print \$$ispos}")
     echo "selected instruction set: $INSTR_SET"
@@ -429,23 +468,13 @@ if [ $FREIFUNK_OPENWRT_BASE == "master" ]; then
     FREIFUNK_OPENWRT_BASE="snapshots"
 fi
 
-if [ -z "$CONF_RELEASE" ] && [ -z "$CONF_TARGET" ]; then
-    # build all targets for all releases
-    for release in $RELEASES; do
-        for target in $(fetch_subdirs $release); do
-            for subtarget in $(fetch_subdirs $release$target); do
-                imagebuilder=$(fetch_subdirs $release$target$subtarget | grep imagebuilder)
-                start_build $release$target$subtarget$imagebuilder
-            done
-        done
-    done
-    exit
-elif [ -z "$CONF_TARGET" ]; then
+
+if [ -z "$CONF_TARGET" ] && [ -z "$IMAGE_BUILDER_PATH" ]; then
     # build one release for all targets
     RELEASE_LINK="$RELEASE_LINK_BASE""$FREIFUNK_OPENWRT_BASE""/targets/"
     for target in $(fetch_subdirs "$RELEASE_LINK"); do
         for subtarget in $(fetch_subdirs $RELEASE_LINK$target); do
-            imagebuilder=$(fetch_subdirs $RELEASE_LINK$target$subtarget | grep imagebuilder)
+            imagebuilder=$(fetch_subdirs $RELEASE_LINK$PARSER_PROFILE$target$subtarget | grep imagebuilder)
             start_build $RELEASE_LINK$target$subtarget$imagebuilder
         done
     done
@@ -454,7 +483,7 @@ else
     # there was given a release and a target
     RELEASE_LINK="$RELEASE_LINK_BASE""$FREIFUNK_OPENWRT_BASE""/targets/"
     # if there was defined a subtarget and option device, only build that.
-    if [ -n "$CONF_SUBTARGET" ]; then
+    if [ -n "$CONF_SUBTARGET" ] || [ -n "$IMAGE_BUILDER_PATH" ]; then
         # build directly that subtarget. if requested, for all image types.
         TARGET_LIST="$RELEASE_LINK$CONF_TARGET/$CONF_SUBTARGET/"
         IMAGEBUILDER=$(fetch_subdirs "$TARGET_LIST" | grep imagebuilder)
@@ -466,6 +495,7 @@ else
             done
         else
             echo "BUILDING 1 PACKAGLIST"
+            # "targets" is on purpose there. Otherwise that positonal argument would be empty.
             start_build "$TARGET_LIST$IMAGEBUILDER" targets $CONF_DEVICE
         fi
         exit