Establish github workflow to build git-annex on OSX

[yarikoptic]

Probably based on what we have for .deb already: https://github.com/datalad/datalad-extensions/blob/master/.github/workflows/build-git-annex-debianstandalone.yaml . Here are general instructions: https://git-annex.branchable.com/install/fromsource/ . May be @joeyh could complement if anything custom is needed based on his current setup of daily builds.

Besides daily builds eventually we might want to establish dedicated builds for tagged releases of git-annex and deposition of artifacts so they could possibly become the "official" builds for git-annex (unless @joeyh wants to do it elsewhere/differently).

Given that we will soon start working with datalad in DANDI, availability of OS X environment, knowledge of Haskell, @jwodder -- could you please take care about this one?

If there is a sensible way to RF aforementioned build-git-annex-debianstandalone.yaml so it builds/tests on both Debian and OSX -- would be great. If not -- may be there is a way to make at least testing setup reused somehow.

[jwodder]

Is the goal here to produce just a standalone executable or a .pkg installer file or something else?

Also, are you aware that git-annex can be installed on macOS with Homebrew? I don't suppose that obviates the need for this issue, does it?

EDIT: I just found this page: https://git-annex.branchable.com/install/OSX/. Where are the builds/build steps for the given .dmg now?

[yarikoptic]

yes, I know that homebrew has a build of annex.

The purpose here is somewhat twofold:

1. replace a builder of an "app bundle" (.dmg) which is ATM done on some elderly iMac I have somewhere in the office and which shows all signs of age and might die sooner than later.
2. add testing against datalad by installing freshly built "app bundle" in that OSX environment and running datalad tests (as we do for Debian Linux ATM)

all build steps seems to be within Makefile (osxapp target) and sample build log shows them: https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/log.cgi?log=3a9d6a2e12a2cd02c438d9e70bffb2bc39c96b61

[yarikoptic]

re installing .dmg from cmdline -- at some point I scripted this helper https://github.com/yarikoptic/datalad/blob/HEAD/tools/upgrade-annex-osx.sh which might be of help?

[jwodder]

Based on my reading of the build log, only the part under "+ make osxapp" seems to be part of the Makefile; the rest of the commands appear to belong to some shell script that sets some things up before running the Makefile. Where can I find this shell script?

[yarikoptic]

indeed, could not find any hit for export OSX_MAGIC_FILE= within git-annex git repo.
ok -- managed to login (as myself, not as joey) to that OSX box! grepped my way into

/Users/joey/gitannexbuilder/build.sh

#!/bin/bash -x
#
# Copy this file to build.sh so that gitbuilder can run it.
#
# What happens is that gitbuilder will checkout the revision of your software
# it wants to build in the directory called gitbuilder/build/.  Then it
# does "cd build" and then "../build.sh" to run your script.
#
# You might want to run ./configure here, make, make test, etc.
#

REV="$(git rev-parse HEAD)"
OUTDIR="../out/output/sha1/$REV"
OUTDIR_TMP="${OUTDIR}.tmp"
install -d -m0755 -- "$OUTDIR_TMP"
printf '%s\n' "$REV" >"$OUTDIR_TMP/sha1"

# Enable build flags that need extra C libs, so are not enabled by
# default. Needs this run to install them:
#       brew install --bottle libmagic pkg-config
export BUILDEROPTIONS="--extra-include-dirs=$HOME/homebrew/include --extra-lib-dirs=$HOME/homebrew/lib"
export OSX_MAGIC_FILE="$(brew list --verbose libmagic| grep magic.mgc | head -n 1)"
cat standalone/osx/stack.yaml > stack.yaml.autobuild
export BUILDERCOMMONOPTIONS="--stack-yaml stack.yaml.autobuild"

export BUILDER=stack
stack setup --stack-yaml stack.yaml.autobuild
# use stack version of ghc
GHC="stack --stack-yaml stack.yaml.autobuild ghc --"
export GHC

$GHC --make Build/BuildVersion
Build/BuildVersion > $OUTDIR_TMP/build-version

make osxapp || exit 4
cp -a tmp/git-annex.dmg $OUTDIR_TMP
./git-annex version

if ! make test > testlog 2>&1; then
        cat testlog
        exit 1
fi
rm -f testlog

rm -f stack.yaml.autobuild
rm -f stack.yaml.autobuild.lock

# put our temp files inside .git/ so ls-files doesn't see them
git ls-files --modified >.git/modified-files
if [ -s .git/modified-files ]; then
    rm -rf "$OUTDIR_TMP"
    echo "error: Modified files:" 1>&2
    cat .git/modified-files 1>&2
    exit 6
fi

git ls-files --exclude-standard --others >.git/added-files
if [ -s .git/added-files ]; then
    rm -rf "$OUTDIR_TMP"
    echo "error: Added files:" 1>&2
    cat .git/added-files 1>&2
    exit 7
fi

# we're successful, the files are ok to be published; try to be as
# atomic as possible about replacing potentially existing OUTDIR
if [ -e "$OUTDIR" ]; then
    rm -rf -- "$OUTDIR.old"
    mv -- "$OUTDIR" "$OUTDIR.old"
fi
mv -- "$OUTDIR_TMP" "$OUTDIR"
rm -rf -- "$OUTDIR.old"

exit 0

@joeyh - I hope it is ok that I share it. I have not spotted any secrets to be mindful of.

@jwodder - email is coming with instructions how to login to that OSX box if you need more information

[jwodder]

For converting the "crippled" tests to macOS, there doesn't seem to be a way to specifically create a VFAT image on macOS, but I can create an image that's FAT, FAT12, FAT16, FAT32, or ExFat. Are any of those acceptable?

[yarikoptic]

I bet any of those will be crippled enough! Thank you!