Skip to content

Commit

Permalink
update.sh: simplify, remove nix-prefetch
Browse files Browse the repository at this point in the history
  • Loading branch information
colemickens committed Dec 27, 2020
1 parent fd77b9e commit 44ac2c5
Show file tree
Hide file tree
Showing 7 changed files with 181 additions and 135 deletions.
4 changes: 3 additions & 1 deletion .ci/prep.sh
Expand Up @@ -10,6 +10,8 @@ git config --global user.name \
git config --global user.email \
"cole.mickens+colebot@gmail.com"

git status

# first things first, let's update our flake
nix --experimental-features 'nix-command flakes' flake update --recreate-lock-file --no-registries
(git add -A . && git commit -m "auto-update: flake.lock") || true
(git add -A . && git commit -m "auto-update(${JOB_ID}): flake.lock") || true
3 changes: 2 additions & 1 deletion .ci/srht-job.yaml
Expand Up @@ -6,11 +6,12 @@ secrets:
- bd070f2a-01d1-4b7b-9795-036deb06912c # id_ed25519
environment:
CI_BUILD: "sr.ht"
FORCE_RECHECK: "true"
tasks:
# prep in a non-pure nix-shell, relies on git+openssh+cacert at least
- prep: |
cd nixpkgs-wayland
nix-shell -I 'https://github.com/nixos/nixpkgs/archive/nixos-unstable.tar.gz' -p nixUnstable --command "./.ci/prep.sh"
- update: |
cd nixpkgs-wayland
nix-shell --pure --command "JOB_ID=$JOB_ID ./update.sh"
nix-shell --pure --keep JOB_ID --keep FORCE_RECHECK --keep CI_BUILD --command "./update.sh"
6 changes: 3 additions & 3 deletions README.md
Expand Up @@ -168,9 +168,7 @@ nix-env -iA neatvnc

These packages were mostly recently built (and cached) against:

<!--pkgs-->
| Package | Description |
| ------- | ----------- |
<!--pkgs-->\n| Package | Description |\n| --- | --- |
| [aml](https://github.com/any1/neatvnc) | liberally licensed VNC server library that's intended to be fast and neat |
| [cage](https://www.hjdskes.nl/projects/cage/) | A Wayland kiosk |
| [clipman](https://github.com/yory8/clipman) | A basic clipboard manager for Wayland, with support for persisting copy buffers after an application exits |
Expand All @@ -180,6 +178,7 @@ These packages were mostly recently built (and cached) against:
| [grim](https://github.com/emersion/grim) | Grab images from a Wayland compositor |
| [gtk-layer-shell](https://github.com/wmww/gtk-layer-shell) | A library to create panels and other desktop components for Wayland using the Layer Shell protocol |
| [i3status-rust](https://github.com/greshake/i3status-rust) | Very resource-friendly and feature-rich replacement for i3status |
| [imv](https://github.com/eXeC64/imv) | A command line image viewer for tiling window managers |
| [kanshi](https://github.com/emersion/kanshi) | Dynamic display configuration |
| [lavalauncher](https://git.sr.ht/~leon_plickat/lavalauncher) | A simple launcher for Wayland. |
| [libvncserver_master](https://libvnc.github.io/) | VNC server library |
Expand All @@ -198,6 +197,7 @@ This plugin only works with obs-studio-dmabuf. |
| [swayidle](https://swaywm.org) | Sway's idle management daemon |
| [swaylock](https://swaywm.org) | Screen locker for Wayland |
| [sway](https://swaywm.org) | i3-compatible tiling Wayland compositor |
| [waybar](https://github.com/Alexays/Waybar) | Highly customizable Wayland Polybar like bar for Sway and Wlroots based compositors. |
| [wayfire](https://wayfire.org/) | 3D wayland compositor |
| [waypipe](https://gitlab.freedesktop.org/mstoeckl/waypipe/) | Network transparency with Wayland |
| [wayvnc](https://github.com/any1/wayvnc) | A VNC server for wlroots based Wayland compositors |
Expand Down
18 changes: 9 additions & 9 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion flake.nix
Expand Up @@ -126,7 +126,7 @@
++ (with pkgs_.cachix.${system}; [ cachix ])
++ (with pkgs_.nixpkgs.${system}; [
nixUnstable nix-prefetch nix-build-uncached
bash cacert curl git jq mercurial openssh ripgrep
bash cacert curl git jq mercurial openssh ripgrep parallel
])
;
}
Expand Down
134 changes: 134 additions & 0 deletions pkgs/update.sh
@@ -0,0 +1,134 @@
#!/usr/bin/env bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
set -euo pipefail
set -x

unset NIX_PATH

# build up commit msg
cprefix="auto-update(${JOB_ID:-"manual"}):"

# keep track of what we build for the README
pkgentries=(); nixpkgentries=();
cache="nixpkgs-wayland";

nixargs=(--experimental-features 'nix-command flakes')
buildargs=(
--option 'extra-binary-caches' 'https://cache.nixos.org https://nixpkgs-wayland.cachix.org'
--option 'trusted-public-keys' 'cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= nixpkgs-wayland.cachix.org-1:3lwxaILxMRkVhehr5StQprHdEo4IrE8sRho9R9HOLYA='
--option 'build-cores' '0'
--option 'narinfo-cache-negative-ttl' '0'
)

## the rest should be generic across repos that use `update.sh`+`metadata.nix`

if [[ "${1:-}" == "updateinternal" ]]; then
##
## internal script (called in parallel)
##
shift
t="$(mktemp)"; trap "rm ${t}" EXIT;
m="$(mktemp)"; trap "rm ${m}" EXIT;
l="$(mktemp)"; trap "rm ${l}" EXIT;
pkg="${1}"
metadata="${pkg}/metadata.nix"
pkgname="$(basename "${pkg}")"

if [[ ! -f "${pkg}/metadata.nix" ]]; then exit 0; fi

nix "${nixargs[@]}" eval -f "${metadata}" --json > "${t}" 2>/dev/null
branch="$(cat "${t}" | jq -r .branch)"
rev="$(cat "${t}" | jq -r .rev)"
sha256="$(cat "${t}" | jq -r .sha256)"
upattr="$(cat "${t}" | jq -r .upattr)"; # optional, but set if not user-set
if [[ "${upattr}" == "null" ]]; then upattr="${pkgname}"; fi
url="$(cat "${t}" | jq -r .url)" # optional
cargoSha256="$(cat "${t}" | jq -r .cargoSha256)" # optional
vendorSha256="$(cat "${t}" | jq -r .vendorSha256)" # optional
skip="$(cat "${t}" | jq -r .skip)" # optional
repo_git="$(cat "${t}" | jq -r .repo_git)" # optional
repo_hg="$(cat "${t}" | jq -r .repo_hg)" # optional

if [[ "${skip}" == "true" ]]; then
echo "skipping (pinned to ${rev})"
exit 0
fi

# grab the latest rev from the repo (supports: git, merucurial)
if [[ "${repo_git}" != "null" ]]; then
repotyp="git";
repo="${repo_git}"
newrev="$(git ls-remote "${repo}" "${branch}" | awk '{ print $1}')"
elif [[ "${repo_hg}" != "null" ]]; then
repotyp="hg";
repo="${repo_hg}"
newrev="$(hg identify "${repo}" -r "${branch}")"
else
echo "unknown repo_typ"
exit 1;
fi

# early quit if we don't need to update
if [[ "${rev}" == "${newrev}" && "${FORCE_RECHECK:-""}" != "true" ]]; then
echo "up-to-date (${rev})"
exit 0
fi

echo "${rev} => ${newrev}"

# Update Sha256
sed -i "s|${rev}|${newrev}|" "${metadata}"; echo $?
sed -i "s|${sha256}|0000000000000000000000000000000000000000000000000000|" "${metadata}"
nix "${nixargs[@]}" build "..#${upattr}" &> "${l}" || true
newsha256="$(cat "${l}" | grep 'got:' | cut -d':' -f2 | tr -d ' ' || true)"
if [[ "${newsha256}" == "sha256" ]]; then newsha256="$(cat "${l}" | grep 'got:' | cut -d':' -f3 | tr -d ' ' || true)"; fi

newsha256="$(nix "${nixargs[@]}" to-sri --type sha256 "${newsha256}")"
sed -i "s|0000000000000000000000000000000000000000000000000000|${newsha256}|" "${metadata}"

# CargoSha256 has to happen AFTER the other rev/sha256 bump
if [[ "${cargoSha256}" != "null" ]]; then
sed -i "s|${cargoSha256}|0000000000000000000000000000000000000000000000000000|" "${metadata}"
nix "${nixargs[@]}" build "..#${upattr}" &> "${l}" || true
newcargoSha256="$(cat "${l}" | grep 'got:' | cut -d':' -f2 | tr -d ' ' || true)"
if [[ "${newcargoSha256}" == "sha256" ]]; then newcargoSha256="$(cat "${l}" | grep 'got:' | cut -d':' -f3 | tr -d ' ' || true)"; fi
newcargoSha256="$(nix "${nixargs[@]}" to-sri --type sha256 "${newcargoSha256}")"
sed -i "s|0000000000000000000000000000000000000000000000000000|${newcargoSha256}|" "${metadata}"
fi

# VendorSha256 has to happen AFTER the other rev/sha256 bump
if [[ "${vendorSha256}" != "null" ]]; then
sed -i "s|${vendorSha256}|0000000000000000000000000000000000000000000000000000|" "${metadata}"
nix "${nixargs[@]}" build "..#${upattr}" &> "${l}" || true
newvendorSha256="$(cat "${l}" | grep 'got:' | cut -d':' -f2 | tr -d ' ' || true)"
if [[ "${newvendorSha256}" == "sha256" ]]; then newvendorSha256="$(cat "${l}" | grep 'got:' | cut -d':' -f3 | tr -d ' ' || true)"; fi
newvendorSha256="$(nix "${nixargs[@]}" to-sri --type sha256 "${newvendorSha256}")"
sed -i "s|0000000000000000000000000000000000000000000000000000|${newvendorSha256}|" "${metadata}"
fi

# Commit
git diff || true
git diff-index --quiet HEAD "${pkg}" || true
git diff-index --quiet HEAD "${pkg}" || \
git commit "${pkg}" -m "${cprefix} ${pkgname}: ${rev} => ${newrev}"
echo "done updating ${pkg} (${rev} => ${newrev})"
exit 0
fi

##
## main script
##

# updates galore
pkgslist=()
for p in `ls -v -d -- ./*/ | sort -V`; do
pkgslist=("${pkgslist[@]}" "${p}")
done

# collect package names into array, pipe into parallel
_nproc="$(nproc)"
_nproc="1"
echo "====> starting internal package update (p=${_nproc})"
printf '%s\n' "${pkgslist[@]}" | \
parallel --jobs "${_nproc}" --halt soon,fail=1 --tag -- "${0} updateinternal '{.}'"
echo "====> done with internal package updates"

0 comments on commit 44ac2c5

Please sign in to comment.