Put in place a process for handling 'promoted' (and 'relegated') devices

[petefoth]

A device is 'promoted' when LineageOS move it from one LOS version to a higher one in their build target list e.g. from 19.1 to 20.0.

A device is 'relegated' when LineageOS remove it from their build target list

Promoted devices

What should happen for promoted builds is

1. At first, we keep one build - the most recent - of the old version 19.1, alongside our new 20.0 build(s).
2. When we are happy that the new versions works OK, then we should remove the old version (19.1) build and keep only new version builds.

At the moment, 1. is (should be) handled when build.sh calls clean_up.py : that should keep $DELETE_OLD_ZIPS" (currently 3) builds of the new version, and 1 build of any old versions that are currently present.

As it stands - even if that is working as designed - it means we are keeping old builds around long after they are obsolete.For example after this month's build run, for some devices we had 18.1, 19.1 & 20.1 builds.

So we need a process to implement 2. The process needs to specify:

1. How do we know that a device has been promoted
2. How do we determine that the new version is OK and it is safe to remove old versions
3. How the old version build artefacts are removed

Same parts of the process can / should be automated, but other parts - specifically removal of the old version build artefacts - should have a manual check before they are performed.

Relegated devices

What should happen for relegated builds is

• To be decided: we could keep a single build - the most recent, or we could remove all builds after a decent interval. What we choose may depend on available disk space

[petefoth]

1. We can tell when devices are promoted
   • by comparing the build target list for the current run to the previous target list for the current run
   • by monitoring / subscribing to changes to the LOS build target list

[petefoth]

DRAFT: work in progress, to be continued....

Build Retention

Strategy

When a device is 'new', only one build is available, on both Download server (DS) and Build server (BS)

While a device is supported by LOS, two builds will available on DS

• Normally, both builds will be of the currently supported branch
• When a device is first 'promoted' to a higher branch (e.g.c19.1 -> 20.0), we have two builds
  • one build on the DS is of the higher branch and stays in zips/<device>
  • one build - the final build of the old branch (e.g. 19.1) - is available in the archive directory on the BS (in case of 'emergencies'). It is kept until the device is promoted again (e.g. 20.0 -> 21.0), when it will be replaced by the final build of the higher branch e.g. 20.0

When support is dropped for a device

• the final two builds are available on the DS, in the normal device directory, for a limited time (i.e. until the next build run is performed).
• after that time, one build (the final build for the device) is available in the archive directory on both the DS and the BS

Use cases to handle when building

1. Device is new this build
   • create newlastbuild marker
   • rsync with delete args, leaving one build on both servers
2. Device was new last build
   • rsync without delete args, leaving two builds on both servers
   • remove newlastbuild marker
3. Current build branch is the same as the previous build branch - the normal, default case
   • rsync with delete args,
4. Device is promoted this build
   • rsync with delete args,leaving two builds - one of each branch - on both servers
   • after rsync
     • remove any existing files from the BS archive directory
     • move (copy & delete) old branch build to BS archive directory, leaving only one build in BS zips directory
5. Device was promoted last build
   • rsync without delete args, leaving two builds on both servers
6. Device is dropped this build
   • remove any existing files from the BS archive directory
   • on both servers
   • remove the penultimate build from the zips directory
   • move the zips directory to the archive directory

Cases 1-6 can be handled per device in 'pre-' or post-build.sh

Case 6 must be handled differently

---

Old ideas

The fix for this will be in our user scripts (pre-build.sh & post-build.sh). These files are not part of this project, but I will handle the issue here in the hope of getting some more 'eyes' on the proposed fix.

Assumptions:

1. When a device is first promoted there will be one or two builds of an earlier branch than the current build. In this case, when the current build of the new branch completes successfully,
   • we should keep one build of the earlier branch. This should be handled OK by the existing 'clean-up.py`
   • in our post_build.sh
     • we can remove any builds that are already in the archive directory for the device,as we only want to keep the latest 'known good' build. rm -r zips/archive/"$Device" && mkdir zips/archive/"$Device"should do what we need
     • we should
       • move the latest build of the earlier branch fromzips/"$device" to the zips/archive/ directory for the device, just in case problems are discovered later with the new branch
       • EITHER sync the zips/archive/$device to the download server so the old branch build is available there.
       • OR just keep the old build on the build server, and only sync with the download server if users report problems with a new build. This might be preferable, as space on the download server is still short .Then the archive directory on the download server will only contain builds - the final build - for device which are no longer supported
     • this will leave only the latest build, of the new branch, in zips/"$device" on both the build and download server
2. The second time we build the new branch, there will only be a single build (of the current) branch. We want to
   • keep this existing build and the build currently being made. This should be handled OK by the existing 'clean-up.py`
   • in our post-build.sh
     • remove the --delete... arguments to rsync so that both builds will be kept on both the build and archive servers
     • remove the old build from zips/archive/"$DEVICE" on the download server (if we chose to copy it across in 2. We may not need to remove it from the build server, where there is (at the moment at least) no shortage of disk space.
     • this will leave the two most recent builds, of the new branch, in zips/"$DEVICE" on both the build and download server
     • EITHER remove zips/archive/$DEVICE on the build server, which currently contains the final build of the old branch
     • OR leave the final build of the old branch there, until the next time the device gets promoted - see point 1 above

Possible logic

1. In pre-build.sh, check if the build is promoted. It is promoted if
   • the zips directory contains one or two builds of an earlier branch than the current build
   • if the device is promoted, then
     • if the directory zips/archive/$DEVICE already exists, then remove all the files it contains
     • else create the directory zips/archive/$DEVICE
     • move the latest build of the earlier branch

[petefoth]

Wed 28 Feb 24

• Strategy implemented in new post-build.sh, committed to my Codeberg repo

`post-build.sh`

#!/bin/sh

DEVICE="$1"
BUILD_SUCCESSFUL="$2"

if [ "$BUILD_SUCCESSFUL" = true ]; then
  cd "$ZIP_DIR/$DEVICE" || return

  # default values - the normal case
  promoted_this_build=false
  rsync_delete_args="--delete --delete-excluded"

  # how many builds do we have now for this device?
  builds_in_zipdir=$(ls -1 lineage-*"$DEVICE".zip | wc -l)
  builds_of_this_branch=$(ls -1 "$BRANCH_NAME"*"$DEVICE".zip | wc -l)

  #if we only have one, rsync without delete args
  if [ "$builds_in_zipdir" = 1 ]; then
    rsync_delete_args=""
  fi

  # a new device
  if [ "$builds_in_zipdir" = 1 ] && [ ! -d "/srv/zips/$DEVICE/" ]; then
    echo "$DEVICE is a new device"
    touch new_last_build

  # device was new the last time we built
  elif [ -f new_last_build ]; then
    echo "$DEVICE was new last time we built"
    rm new_last_build

  # device is newly promoted
  elif [ "$builds_in_zipdir" != "$builds_of_this_branch" ]; then
    promoted_this_build=true
    echo "$DEVICE is promoted to $BRANCH_NAME"

  # device was promoted last time we built
  elif [ "$builds_in_zipdir" = 1  ] && [ -d "/srv/zips/$DEVICE/" ]; then
    echo "$DEVICE was promoted last time we built"
  fi

  # now we can call rsync
  # Copy every file with .zip, .md5sum, .prop or .sha256sum extension.
  # Do not recurse into directories
  # optionally remove from the destination every file
  # not present in the source or not ending with the allowed extensions.
  # Keep the original modification timestamp.
  rsync -e "ssh -i \"$SCRIPT_PATH/rsync_key\" -o UserKnownHostsFile=\"$SCRIPT_PATH/rsync_known_hosts\"" \
      --verbose --dirs "$rsync_delete_args" --times \
      --include="*.zip" --include="*.img" --include="*.md5sum" --include="*.sha256sum" --include="*.prop"\
      --exclude="*" \
      "/srv/zips/$DEVICE/" "$HOST:zips/$DEVICE"

  if [ $promoted_this_build = true ]; then
    # remove any files in the archive directory
    archive_dir="/srv/zips/archive/$DEVICE/"
    rm -f "$archive_dir" && mkdir "$archive_dir"
    # move the old branch files to the archive
    find . -type f -not -name "$BRANCH_NAME*" -exec mv '{}' "$archive_dir" \;
  fi
  $(dirname $0)/matrix.sh "Completed build for $1"
else
 	$(dirname $0)/matrix.sh "Failed build for $1"
fi

[petefoth]

Test the new post-build.sh

• z3tc/w 18.1- tests 'new device
  • create manifest l4m_build_scripts/manifests/18.1/l4m-sony-z3tc-18.xml
• maple 20 - new last time - need to touch new_last_build inzips/maple
• poplar 21 - promoted
  • create manifest l4m_build_scripts/manifests/21./l4m-sony-yoshino-20.xml
• make unofficial builds in file test-build.sh

./build_unofficial.sh lineage-18.1 z3tcw
./build_unofficial.sh lineage-20.0 maple
./build_unofficial.sh lineage-21.0 poplar

• check spoofing is set tp no in all build files
• run test-build.sh
cd /home/docker-build/l4m_build_scripts && ./test-build.sh 2>&1 | tee -a /home/docker-build/logs/test-build-$(date '+%Y%m%d').log

[petefoth]

• can't make it work - use the old post-build.sh (with --delete) for this build run
• commands to list what branch was built for each device

#!/bin/sh

BUILD_MONTH="$1"

cd /mnt/persistent/zips

for device in $(ls -1);  do
  find "$device" -type f -name "lineage*202402*.zip" | sed 's/\/lineage-/ /' | sed 's/-.*$//'
done > "/home/docker-build/logs/build-branches-$BUILD_MONTH.txt"
cat "/home/docker-build/logs/build-branches-$BUILD_MONTH.txt"
cd